Hi Vincent, I actually replied to this yesterday from my phone after you posted but it seemed to get lost in the ether. No comments about my phone Vincent! ;-) Really well done on finding this out as the '#}' message was the major stumbling block I was having during my investigations. I guess I was right in my assumptions that it was having issues with our site but it's good that it's been rectified with such a simple change. Robert
> Date: Fri, 25 Jul 2014 10:26:57 +0100 > From: [email protected] > To: [email protected] > Subject: Re: FOP Release Automation > > Hi Vincent, > > On 24/07/2014 17:45, Vincent Hennebert wrote: > > [Moving to general@ as other sub-projects can benefit from this too.] > > > > I thought I would have a go at it and believe I’ve found a solution. > > There was a problem of conflicting syntaxes between Markdown’s way of > > specifying custom header IDs, and Dotiac::DTL’s syntax for comments. > > That's great news. Well done Vincent! Now we just need someone to add > all the variables we need and write a script to change the variables at > release time. > > > > > For example: > > > > ## Project Status {#status} > > > > The ‘{#status}’ is Mardown’s way of giving a custom ID to the generated > > H2 element. But ‘{#’ is also a way to introduce a comment in > > a Dotiac::DTL template, which then complains that there is no closing > > ‘#}’. > > > > Fortunately, both can be made to agree by inserting a space between ‘{’ > > and ‘#’: > > > > ## Project Status { #status} > > > > The Dotiac:DTL comment has now gone, but Markdown still recognises it as > > a custom ID. (A lot of them are actually unnecessary since they are > > identical to the title, and would be automatically generated by the > > Markdown HeaderId extension. They come from the conversion of Forrest > > sources, at some point we may want to scan through and remove them.) > > > > As you may have noticed I’ve just committed a batch change of header IDs > > that introduces that space. Now we can enable pre-processing of the > > Markdown sources and enjoy the use of variables (and much more if you are > > daring). > > > > Fortunately there’s hardly any Perl involved. We just need to set some > > parameters in the path.pm and define variables there. See > > http://svn.apache.org/r1613178 for more details. I’ve added just two > > variables as a proof of concept but now we can proceed scanning the > > sources and adding more variables. > > > > I don’t know whether there’s a risk that our variables conflict with the > > CMS’ own variables. If we name them properly (e.g., prefixing them with > > fop_ or batik_ or xgc_), then this should not happen. > > > > Vincent > > Thanks, > > Chris > > > > > > > On 15/07/14 11:35, Robert Meyer wrote: > >> Hi All, > >> > >> This is an update into my investigations on automating the release > >> process for > >> FOP. As we're nearing release it looks as though version 2.0 will > >> remain a > >> manual process for the time being. That's not to say that it will > >> forever > >> remain like that but at present unless a breakthrough occurs or I > >> receive some > >> feedback from the infrastructure team, I don't currently have the > >> necessary > >> knowledge on the Apache infrastructure (or Perl know how) to achieve the > >> desired result despite my efforts. > >> > >> During the time since my last message I found a solution using a > >> markdown > >> extension. This appeared to fulfil all of our requirements and after > >> writing > >> and testing one, it seemed to simply be a case of installing it. Due > >> to the > >> nature of Apache's websites this was not something I could do myself > >> as we > >> don't have control over the CMS. After raising a ticket with the > >> infrastructure team about doing this, I was pointed to another > >> project called > >> Thrift. Their site appeared to provide tag replacement using preexisting > >> functionality found in the perl modules of the Apache CMS. > >> > >> After reading the documentation and experimenting I've reached > >> somewhat of an > >> impasse due to a number of reasons. Firstly the documentation on > >> customizing > >> these patterns is limited and covers only basic patterns. Second, my own > >> experience with Perl is lacking and as such makes it hard to debug and > >> understand some of the more complicated required modules and sections > >> of the > >> CMS. Finally during my testing the errors I was getting were extremely > >> unhelpful and provide next to no clues as to where the problem lay in > >> my own > >> code. Instead they point to the Perl CMS libraries highlighting missing > >> expected characters and at a guess incompatibilities between the > >> markdown > >> we're using and what's expected by the pattern's own subroutine. > >> > >> I have tried to follow and utilize the code found in the Thrift > >> project with > >> little luck. I have posted on the infrastructure mailing list for > >> help but as > >> of yet have not had any responses. I am guessing this is not a > >> commonly used > >> feature and as such knowledge on the subject may be in short supply. > >> As such > >> and without possibility of using the markdown extension we're left > >> with the > >> manual process for the time being. I will keep an eye out on the > >> infrastructure page and prod them occasionally to see if I can move > >> things along. > >> > >> Apologies for the long e-mail but just wanted to keep you all updated. > >> > >> Robert Meyer > >> > >> > Date: Mon, 2 Jun 2014 14:44:58 +0100 > >> > From: [email protected] > >> > To: [email protected] > >> > Subject: Re: FOP Release Automation > >> > > >> > Hi All, > >> > > >> > I certainly use the web interface when making small tweaks to the > >> docs. > >> > As you know users occasionally report small mistakes that require > >> minor > >> > tweaks. I'd like to streamline the updating of the website for > >> release > >> > purposes but I don't want to disable/prevent the current web > >> > interface which works well when all you want to do is make a minor > >> > adjustment in response to a user e-mail. > >> > > >> > Rob is away this week, but he will continue the investigation into > >> > scripting the website updates when he returns. > >> > > >> > Thanks for the ideas so far, a few promising leads. > >> > > >> > Thanks, > >> > > >> > Chris > >> > > >> > On 30/05/2014 17:23, Clay Leeds wrote: > >> > > Agreed, ‘some’ people wouldn’t be happy with that. ;-) > >> > > > >> > > I wonder if the CMS Web interface could be extended to allow for > >> a few > >> > > keywords like FOP_VERSION, FOP_REVISION, FOP_BRANCH, etc. > >> > > > >> > > The CMS tool's WYSIWYG interface indicates it uses the Wysiwym > >> > > MarkDown Editor, which is extensible: > >> > > > >> > > https://web.archive.org/web/20110121060932/http://wmd-editor.com/ > >> > > > >> > > (site’s down & hasn’t been updated since 2011)... > >> > > > >> > > or > >> > > > >> > > https://code.google.com/p/wmd/ > >> > > > >> > > We might still need to do some ANT hanky panky, but at least if we > >> > > could leverage WMD’s extensibility it would help us get where we’re > >> > > trying to go? > >> > > > >> > > Clay > >> > > > >> > > On May 30, 2014, at 7:19 AM, Robert Meyer <[email protected] > >> > > <mailto:[email protected]>> wrote: > >> > >> Hi, > >> > >> > >> > >> I like the simplicity of your idea, but the web interface is > >> not so > >> > >> easy to dismiss unfortunately. > >> > >> > >> > >> If you do have a copy with those tags in, if any changes are > >> made on > >> > >> the web interface then that copy would become out of date. > >> > >> > >> > >> We could always shutdown the web interface, but I don't think too > >> > >> many people would be happy with that ;-) > >> > >> > >> > >> Regards, > >> > >> > >> > >> Robert > >> > >> > >> > >> > >> ------------------------------------------------------------------------ > >> > >> From: [email protected] > >> <mailto:[email protected]> > >> > >> To: [email protected] > >> > >> <mailto:[email protected]> > >> > >> Subject: RE: FOP Release Automation > >> > >> Date: Fri, 30 May 2014 14:48:15 +0100 > >> > >> > >> > >> Hi, > >> > >> > >> > >> Simple way is to store docs inside fop repo: > >> > >> > >> > >> Fop/docs/index.markdown > >> > >> > >> > >> Inside markdown file you reference ant properties eg: > >> > >> ${version} > >> > >> > >> > >> Then you call which does ant expandproperties and calls > >> markdown to > >> > >> html tool: > >> > >> ant docs > >> > >> > >> > >> Then you call which does a zip, scp and unzip of html files to web > >> > >> server: > >> > >> ant upload-docs > >> > >> > >> > >> This method doesn’t support web interface of editing files but I > >> > >> don’t see how this is really needed. > >> > >> If I submit a patch to fop it should also contain doc changes > >> rather > >> > >> than having separate repo and patch for that. > >> > >> > >> > >> Thanks > >> > >> > >> > >> *From:*Robert Meyer [mailto:[email protected]] > >> > >> *Sent:*30 May 2014 14:05 > >> > >> *To:*[email protected] > >> > >> <mailto:[email protected]> > >> > >> *Subject:*RE: FOP Release Automation > >> > >> > >> > >> Hi, > >> > >> > >> > >> After investigating your suggestions Clay I have found that > >> svn-hooks > >> > >> can't be used for the purpose we require unfortunately as it > >> may lead > >> > >> to problems with how SVN operates and also may have some > >> unexpected > >> > >> results with files being committed. This is stated in the > >> > >> documentation under "Creating Repository Hooks" highlighted in the > >> > >> warning red box further down: > >> > >> > >> > >> > >> http://www-inst.eecs.berkeley.edu/~cs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html > >> > >> > >> > >> > >> > >> <http://www-inst.eecs.berkeley.edu/%7Ecs61b/fa09/docs/svn-book-html-chunk/svn.reposadmin.create.html> > >> > >> > >> > >> > >> > >> Pascal, I agree that the process is fairly straightforward, but I > >> > >> have been asked to automate this further so am just looking into > >> > >> ideas presently. > >> > >> > >> > >> I think a possible way forward then would be to use your > >> suggestion > >> > >> Pascal of placing the versioned docs for the site inside the FOP > >> > >> repository for their associated version. These can then be > >> referenced > >> > >> using the svn-externals from the main site repository. > >> > >> > >> > >> In addition to this, the main site files (which would need to be > >> > >> updated) could contain tags for the last three versions which > >> would > >> > >> be replaced using Clay's markdown pre-processor suggestion. The > >> > >> pre-processor would replace the tags with values stored in a > >> > >> properties file in the main site repository. > >> > >> > >> > >> To create a release, the user would need to update the > >> svn-external > >> > >> references to account for the new version and update the > >> properties > >> > >> file for tag replacement. When the properties file is pushed it > >> will > >> > >> be read by the custom markdown pre-processor and display the new > >> > >> version when rendered. > >> > >> > >> > >> Those two stages could be done using a single script to simplify > >> > >> things further, but the main complication is getting the markdown > >> > >> pre-processor working. From looking at this page: > >> > >> > >> > >> http://www.apache.org/dev/cmsref.html#markdown > >> > >> > >> > >> I am guessing we use PyPy (Python Markdown) which supports > >> > >> extensions, so I will look at this shortly to try out a small > >> example > >> > >> and investigate the feasibility of doing this. There is also the > >> > >> matter of updating the versioned documents for each FOP when a new > >> > >> version is released, but maybe this could be done with the > >> > >> pre-processor as well. > >> > >> > >> > >> Anyway, let me know what you think. > >> > >> > >> > >> Regards, > >> > >> > >> > >> Robert > >> > > > >> > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
