Re: [leaf-devel] SF shell cron job: doc-build.sh
Bit again by the mail address meant to send to list. Grrr.. Mike Noyes wrote: David, Thanks for the suggestion. I'll run it by one of the SF staff members this week. I'm not hopeful they'll allow this, though. I'm not sure why not. When I say user space, I mean cfengine can run as a regular user program instead of as a system daemon. We could run it, and even use it (as I said before) as a sort of 'cron' just for us. However, it does become a long-running process then - which probably needs the ok. Even so, one could use it on demand if need be - rather than letting it run as a background process. --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
Hello Mike; thx for your work. Am Freitag, 14. April 2006 16:45 schrieb Mike Noyes: On Thu, 2006-04-13 at 20:20, Charles Steinkuehler wrote: Looks good to me...once again, I can't comment on the document building commands, but the overall script looks very robust. Charles, Thank you for helping me generate this script. :-) crontab -e # LEAF project daily documentation build. 35 2 * * * /home/groups/l/le/leaf/admin/doc-build.sh Next task: Mediawiki installation. I want to point out two or three problems: 1) The link in A pdf version is available for printing and off-line reading. points to FRS. 2) Can you shrink the chapters/subchapters on the main page to a minimum level, which will be a lot more readable? 2 and half) I suggest to point to Bering-uClibc docs at the beginning, cause it's branch actively maintained - Bering's latest version will have it's third birthday next month. 3) Do you have opened adding docs and they'll included on a daily basis? we can add man pages and whatever independently. Do we have to respect some yet unkown rules (like adding as xml)? regards kp --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Mon, 2006-04-17 at 11:39, KP Kirchdoerfer wrote: Am Freitag, 14. April 2006 16:45 schrieb Mike Noyes: Thank you for helping me generate this script. :-) crontab -e # LEAF project daily documentation build. 35 2 * * * /home/groups/l/le/leaf/admin/doc-build.sh Next task: Mediawiki installation. I want to point out two or three problems: 1) The link in A pdf version is available for printing and off-line reading. points to FRS. KP, Yes. This is where I plan to place the pdf now. It'll be easier to download from the FRS. 2) Can you shrink the chapters/subchapters on the main page to a minimum level, which will be a lot more readable? Will do during docbook xsl customization layer generation. 2 and half) I suggest to point to Bering-uClibc docs at the beginning, cause it's branch actively maintained - Bering's latest version will have it's third birthday next month. I can reorder as you request. However, I believe our ToC generation had this order before. The only difference is the amount of detail in it. Please let me know how you wish to proceed: A) Wait for customization layer B) Reorder leaf-doc-set.xml 3) Do you have opened adding docs and they'll included on a daily basis? we can add man pages and whatever independently. Do we have to respect some yet unkown rules (like adding as xml)? I want all of our documentation in cvs as docbook xml, except doc/man. The man pages are converted using groff. Not ideal, but groff does work. Note: I'll help with the remaining conversion to docbook xml after I install Mediawiki, and generate our docbook xsl customization layer. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Mon, 2006-04-17 at 11:39, KP Kirchdoerfer wrote: 3) Do you have opened adding docs and they'll included on a daily basis? we can add man pages and whatever independently. Do we have to respect some yet unkown rules (like adding as xml)? KP, I probably wasn't clear in my last post. :-( Feel free to add any man page you wish to doc/man. Also, you can add docbook xml to doc/guide or doc/howto. Note: I'd like to see our network_diagrams converted to svg. Firefox supports svg natively, and so does docbook xsl. http://www.sagehill.net/docbookxsl/SVGimages.html -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Noyes wrote: Everyone, I placed revision 1.12 in cvs yesterday. Please take a look at it if you have a chance. Comments and feedback are welcome. If everyone thinks it's usable, I'll enable it in crontab on Monday. RCS file: /cvsroot/leaf/sourceforge/admin/doc-build.sh,v I finally got a chance to look at this, and the CVS version is only 1.8 (*NOT* 1.12). Looks like the SF CVS issues ate some of the most recent commits. Can you post the latest file for review? - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEPQWJLywbqEHdNFwRApKbAKC6YFHnbbqZFW5u+HH/t3PFNG9CDwCgzcu7 z4VOca5FrnGJPzMj3YGUnI8= =8AXe -END PGP SIGNATURE- --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Sat, 2006-04-08 at 13:05, Mike Noyes wrote:
I placed revision 1.12 in cvs yesterday. Please take a look at it if you
have a chance. Comments and feedback are welcome.
Everyone,
The formatting on this post should be a bit better.
*** Begin doc-build.sh ***
#! /bin/bash
# PROGRAM:
# PURPOSE: Auto-build documentation.
# HISTORY: 2006-03-16 mhnoyes Initial programming (Mike Noyes)
# HISTORY: 2006-03-18 mhnoyes Added lockfile code (Charles Steinkuehler)
# HISTORY: 2006-03-20 mhnoyes Added error check code (Steve Parker)
# http://steve-parker.org/sh/exitcodes.shtml
# HISTORY: 2006-03-29 mhnoyes Added local xml catalog
# HISTORY: 2006-03-30 mhnoyes Incorporated 'set -e' and 'trap' (David Pashley)
# http://www.davidpashley.com/articles/writing-robust-shell-scripts.html
# NOTES: XSL Customization
# use.id.as.filename
# chunker.output.indent
# xsl:attribute-set name=monospace.verbatim.properties
# xsl:attribute name=wrap-optionwrap/xsl:attribute
# xsl:attribute name=hyphenation-character\/xsl:attribute
# /xsl:attribute-set
# NOTES: XML
# olink database
# xhtml validation
#
# VARIABLES
#
set -e
DATE=`date '+%Y-%m-%d %T %Z'`
PROJ_DIR=/home/groups/l/le/leaf
TMP_DIR=/tmp/persistent/leaf
PSERVER=-d:pserver:[EMAIL PROTECTED]:/cvsroot/leaf
DB_XSL=/home/groups/d/do/docbook/htdocs/release/xsl/current
DB_XSL_CUST=/tmp/persistent/leaf/doc/docbook
LOCKFILE=leafcron.lock
export
XML_CATALOG_FILES=/home/groups/l/le/leaf/admin/docbook-xml-4.4/catalog.xml
/etc/xml/catalog
# export XML_DEBUG_CATALOG=
#
# FUNCTIONS
#
cleanup () {
if [ -e $TMP_DIR/doc ]; then
rm -rf $TMP_DIR/doc
fi
}
check_errors () {
if [ ${1} -ne 0 ]; then
echo ERROR # ${1} : ${2}
cleanup
exit ${1}
fi
}
create_lockfile () {
trap rm -f $TMP_DIR/$LOCKFILE; exit INT TERM EXIT
lockfile -r 0 -l 72000 $TMP_DIR/$LOCKFILE || \
check_errors $? Lockfile creation failed.
echo Got lock file: $LOCKFILE
}
export_documents () {
cd $TMP_DIR
cvs $PSERVER -q export -D $DATE -d doc doc || \
check_errors $? CVS export error.
ln -s $DB_XSL/images doc/docbook/images
echo Options +Indexes
AddType 'text/html; charset=UTF-8' html doc/.htaccess
}
validate_documents () {
cd $TMP_DIR/doc
xmllint --noout --xinclude --postvalid --noent leaf-doc-set.xml || \
check_errors $? Document validation failure!
echo Document validation success.
}
build_doc () {
cd $TMP_DIR/doc
xsltproc --xinclude $DB_XSL/xhtml/chunk.xsl leaf-doc-set.xml || \
check_errors $? XSLT failure!
echo DocBook XML XSLT complete.
}
build_man () {
cd $TMP_DIR/doc/man
for f in *; do
groff -t -mandoc -Thtml $f $f.html
groff -t -mandoc -Tps $f $f.ps
ps2pdf $f.ps
rm $f.ps
done
}
replace_old () {
chgrp -R leaf $TMP_DIR/doc
if [ -e $PROJ_DIR/doc ]; then
rm -rf $PROJ_DIR/doc
fi
mv $TMP_DIR/doc $PROJ_DIR/
}
#
# MAIN PROGRAM
#
create_lockfile
cleanup
echo ---
umask 002
echo Exporting documents from CVS.
export_documents
echo ---
echo Validating documents.
validate_documents
echo ---
echo Converting man pages to html and pdf
build_man
echo ---
echo Building Documentation
build_doc
replace_old
rm -f $TMP_DIR/$LOCKFILE
trap - INT TERM EXIT
*** End doc-build.sh ***
--
Mike Noyes mhnoyes at users.sourceforge.net
http://sourceforge.net/users/mhnoyes/
SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs
---
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642
___
leaf-devel mailing list
leaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Thu, 2006-03-30 at 04:41, Charles Steinkuehler wrote: Mike Noyes wrote: Ugh! They went on for quite a while on that thread. I'm not sure what discussed method was finally used.. http://moongroup.com/pipermail/shell.scripting/2005-February/thread.html#9396 The reason they're not just using the -l switch to lockfile is the person who started the thread is wanting to do something different in the script if a stale lockfile is found: http://moongroup.com/pipermail/shell.scripting/2005-February/009410.html Charles, Thanks for explaining. :-) If you're concerned about this, we'd need to jump through the same hoops they're mentioning in the thread. Personally, I think one of the two easy options would work for us: I think my exposure to the SF staff has made me overly concerned with stale locks. They spend a considerable amount of time attending to cvs stale locks. 1) No automatic stale lockfile removal. If anything goes wrong with the script and a lockfile is left on the system, an administrator will have to login, check things out, and clear the lockfile. 2) Lockfile automatically removed after some (long compared to document creation) time. This would handle the cases where the script died for some reason and didn't remove the lockfile, but could potentially cause problems if a previous script is still running for whatever reason. I'd probably suggest going with #1 for now...the worst case is documents don't get automatically generated for a while if an admin doesn't look at things immediately. Agreed. I mentioned the -l switch to lockfile since I wasn't sure if your removal of the lockfile when trying to grab it failed was unintentional, or an attempt at forcibly grabbing it. I was trying to clear the lockfile before exiting in case of failure, just as your example clears the lockfile on success. I didn't do it right though. From what I understand of the -l switch, it will grab the lock file if it's older than the specified time. It does appear to take a few seconds for this to occur though. Not a big concern in a cron script. Note: I'll need to shorten the -l time for my test runs. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Wed, 2006-03-29 at 14:07, KP Kirchdoerfer wrote: Am Mittwoch, 29. März 2006 23:13 schrieb Mike Noyes: Here is an early run of our new document build script. I'm using the default docbook xsl currently. I'll work on a customization layer after initial build testing, etc. are complete. http://leaf-project.org/doc/new/ does that mean changes will be updated by a cron job again? Would be great! KP, Not yet. I'm still working on doc-build.sh. The current revision in cvs is 1.6. Pending: * feedback on shell code in doc-build.sh * addition of man pages * addition of function to move built documentation from /tmp/persistent to our project shell space. In addition some other stuff I'm probably forgetting right now. Three remarks - pdf does still not work, no pb, but can you remove the link until it works? Understood. This is just an initial run. - Can we have the chapters/subchapters less deeper (only 1 and 1.1 not three or four levels deep as it is now)? I'll address this when I start work on the XSL customization layer. - Current version starts with developer guides, I'd like to have Installation Guide's and User Guide's before Developer Guide's - expecting developers having a closer look and more patience than mere user's :) I'll attend to the requested reordering. thx for taking care of the guides/docs. yw -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid0944bid$1720dat1642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Noyes wrote: On Wed, 2006-03-29 at 14:07, KP Kirchdoerfer wrote: Am Mittwoch, 29. März 2006 23:13 schrieb Mike Noyes: Here is an early run of our new document build script. I'm using the default docbook xsl currently. I'll work on a customization layer after initial build testing, etc. are complete. http://leaf-project.org/doc/new/ does that mean changes will be updated by a cron job again? Would be great! KP, Not yet. I'm still working on doc-build.sh. The current revision in cvs is 1.6. Pending: * feedback on shell code in doc-build.sh I finally got a few free minutes to look at this. Your script looks very clean except for one issue with the check_errors function. You're removing the lockfile in this routine, which is required for expected operation if one of the document building commands fails. You are also using the check_errors procedure for testing whether the lockfile generation worked or not. That means if you run this script while another instance is already running, it will not be able to grab the lockfile, but will then forcibly remove the lockfile without owning it (a BadThing :). It looks like the easiest way to handle this is to special case the create_lockfile error checking (ie: put the logic inside the create_lockfile function). If you want the lockfile forcibly grabbed after some amount of time (ie: 8 to 24 hours or so...something that's much longer than document generation should normally take), you can do this with the -l switch, ie: lockfile -l 72000 ... would force removal of any existing lockfile older than 20 hours. - -- Charles Steinkuehler [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEKxcHLywbqEHdNFwRAgdLAJ9aSDUi05ATZ4uCC/qZgQQmyjPSLQCgxmq0 HycMVYykWU01rZ4KdGwcDtw= =kWfM -END PGP SIGNATURE- --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] SF shell cron job: doc-build.sh
On Wed, 2006-03-29 at 15:23, Charles Steinkuehler wrote: Mike Noyes wrote: Pending: * feedback on shell code in doc-build.sh I finally got a few free minutes to look at this. Charles, Thank you for helping. :-) Your script looks very clean except for one issue with the check_errors function. You're removing the lockfile in this routine, which is required for expected operation if one of the document building commands fails. You are also using the check_errors procedure for testing whether the lockfile generation worked or not. That means if you run this script while another instance is already running, it will not be able to grab the lockfile, but will then forcibly remove the lockfile without owning it (a BadThing :). It looks like the easiest way to handle this is to special case the create_lockfile error checking (ie: put the logic inside the create_lockfile function). If you want the lockfile forcibly grabbed after some amount of time (ie: 8 to 24 hours or so...something that's much longer than document generation should normally take), you can do this with the -l switch, ie: lockfile -l 72000 ... would force removal of any existing lockfile older than 20 hours. Ok. I made the changes in doc-build.sh r1.8. I hope I did it correctly. -- Mike Noyes mhnoyes at users.sourceforge.net http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, phpwebsite, phpwebsite-comm, sitedocs --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
