Re: Re-write of documentation tree build process

2006-12-21 Thread John Mandereau
Han-Wen Nienhuys wrote:
 John Mandereau escreveu:
  Before the next release including this patch, may I also add a link to
  the French tutorial in index.html.in?
 
 Sure.
 
 Here are some final remarks, and Jan will have a look too.  If Jan is
 ok, please push to stable/2.10.

Should I send a patch again, or work according to your remarks and push
to stable/2.10 (and possibly master) if Jan agrees?


 I've moved most stuff from GNUmakefile.in:local-WWW-post to
 stepmake/bin/www_post.py.
 
 Good! Can you put them in buildscripts/

Yes, I'll do this.

 (the stepmake bin/ directory isn't used anymore AFAIK.) 

It's still used for add-html-footer.py, but as it needs to be renamed to
add_html_footer.py because import doesn't accept dashes in filenames,
let's move it to buildscripts. Actually, I'd even like to give it a more
accurate name like html_postprocess.py.


  -$(INSTALL) -m 755 -d $(DESTDIR)$(webdir)
  -   cp -a $(outdir)/web-root/ $(DESTDIR)$(webdir)/
  +   cp -a $(outdir)/offline-root/ $(DESTDIR)$(webdir)/
   
 
 I tend to use rsync for this; on MacOS X, cp -a doesn't work.

OK. Speaking of portability, I don't know how symlinks and hard links
are supported on different platforms. I hope it is supposed to work on
all Unix flavors with file systems supporting symlinks and hardlinks;
does it include MacOS X? (according to the Python docs, os.symlink() and
os.link() are only available on Unix.) Similarly, I'm ignorant about
hardlink and symlink support in Cygwin.



  +def build_pages_dict (filelist):
  +Build dictionnary of available translations of each page
  +pages_dict = {}
  +for f in filelist:
  +m = html_re.match (f)
  +if m:
  +g = m.groups()
 
 hmm, of course the code of add_html_footer is a complete mess, but
 you're excused as it is the combined turd of years of negligence of
 add-html-footer.py by Jan and me. Still, you're welcome to clean it up
 a bit if possible.

I've already junked a bit of clearly unused stuff, but I didn't try to
check whether every bit of code is necessary to the HTML files of Lily
docs. For instance, I wonder whether remove_self_ref() is really used.
I'll try to use the Python debugger to find the unused bits of code.

Btw, I'd like to know you opinion about these possible cleanups:

- could we scrap i18n() and move the code to do_file(), as we may add
i18n in do_file too? (I'd like the footer to be translated, for
instance).

- we don't add any custom header, so all stuff using the header
variable could be dropped. 

- what do all these '[.]' in regular expressions mean? If they should
match a dot in a string, why not just use r'\.' ? I've already
experienced that '[.]' is bogus in some cases.

- quoting add_html_footer.py:
 if re.search ('(?i)!DOCTYPE', s) == None:
 doctype = '!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01 
 Transitional//EN\n'
 s = doctype + s
Does the 'EN' means that the page language is English? If so, it should
be modified to handle pages of any language.


 I think the wiki stuff can go

Yes, according to what Graham recently wrote on the LilyPond wiki
thread.


  +#subst = globals ()
  +#subst.update (locals())
  +for k in page_flavors.keys():
  +page_flavors[k] = page_flavors[k] % vars ()
  +
  +# urg
  +# maybe find first node?
  +fallback_web_title = '-- --'
  +# ugh, python2.[12] re is broken.
  +#m = re.match ('.*?title\(.*?\)/title', s, re.DOTALL)
  +m = re.match ('[.\n]*?title([.\n]*?)/title', s)
 
 similarly, we don't bother with python 2.1 and 2.2 anymore.
 Jan, which is the proper one now?

Python 2.3, according to INSTALL. I've already cleaned up some
deprecated functions (e.g string.foo() functions - string methods).
I'll have a deeper look at that.



  diff --git a/stepmake/bin/mirrortree.py b/stepmake/bin/mirrortree.py
  new file mode 100644
  index 000..4bbac88
  --- /dev/null
  +++ b/stepmake/bin/mirrortree.py
[snip]
  +map (os.mkdir, target_dirs)
  +
 
 btw, what happens if target_dirs already exists?

It simply crashes. It shouldn't happen as target_dirs are deleted with
rm -rf before www_post.py is called.


  diff --git a/stepmake/bin/www_post.py b/stepmake/bin/www_post.py
  new file mode 100644
  index 000..a2fd52e
  --- /dev/null
  +++ b/stepmake/bin/www_post.py
[snip]
  +  targets = targets)
  +find_html = subprocess.Popen (['find'] + doc_dirs + ['-name', '*.html'],
  +  stdout = subprocess.PIPE,
  +  bufsize = -1)
  +find_output = find_html.stdout.read ()
  +n = find_html.wait ()
 
 
   os.popen (..command..).read ()
 
 Still more python is to use os.walk(), but it's a little more work.

That can simply be done in mirrortree.hardlink_tree(), which could
return the HTML files list. Let's go for that.

-- 
John Mandereau [EMAIL PROTECTED]



___
lilypond-devel mailing 

Re: Re-write of documentation tree build process

2006-12-21 Thread Juergen Reuter

On Thu, 21 Dec 2006, John Mandereau wrote:


...
 doctype = '!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01 
Transitional//EN\n'
 s = doctype + s
Does the 'EN' means that the page language is English? If so, it should
be modified to handle pages of any language.


No.  The whole string is just the ID of the HTML DTD and thus does not 
change as long as you stick to HTML 4.01 Transitional).  See:

http://www.w3.org/TR/html4/struct/global.html#h-7.2

I am not sure about the exact meaning of the EN within the DTD ID 
string; you probably would have to look into ISO 8879; but that's not 
online available, as far as I know.


Greetings,
Juergen


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel


Re: Re-write of documentation tree build process

2006-12-21 Thread Jan Nieuwenhuizen
John Mandereau [EMAIL PROTECTED] writes:

 Should I send a patch again, or work according to your remarks and push
 to stable/2.10 (and possibly master) if Jan agrees?

Please push to stable/2.10.

 I've already junked a bit of clearly unused stuff, but I didn't try to
 check whether every bit of code is necessary to the HTML files of Lily
 docs. For instance, I wonder whether remove_self_ref() is really used.
 I'll try to use the Python debugger to find the unused bits of code.

 - could we scrap i18n() and move the code to do_file(), as we may add
 i18n in do_file too? (I'd like the footer to be translated, for
 instance).

Yes.

 - we don't add any custom header, so all stuff using the header
 variable could be dropped. 

If it's not too much of a mess, keep the header stuff in.  We might
want to add a header.

 - what do all these '[.]' in regular expressions mean? If they should
 match a dot in a string,

Yes.

 why not just use r'\.' ?

I never want to know how many \ to use in what kind of string.  [.]
fixes that.

 I've already experienced that '[.]' is bogus in some cases.

I'm not aware of that?

 Does the 'EN' means that the page language is English? If so, it should
 be modified to handle pages of any language.

Yes, I think so.

 That can simply be done in mirrortree.hardlink_tree(), which could
 return the HTML files list. Let's go for that.

I have two remarks, but given the state add-html-footer was in, I
hardly feel I can make any demands; you're free to ignore me.

+mirrortree.hardlink_tree (input_roots = doc_dirs,
+  process_dirs = outdir,
+  strip_dir_names = outdir,
+  exclude_dirs = r'(fr|po|out|\w*?-root)(/|$)',
+  process_files = 
'.*[.](?:midi|pdf|png|txt|ly|signature)$|VERSION',
+  exclude_files = 'lily-[0-9a-f]+.*.pdf',
+  target_pattern = target_pattern,
+  targets = targets)

in general, if a function takes much more 3 parameters, you may want
to take a second look why that is.  It may be to be too generic, or
perhaps it tries to do too many things and can be split into several
functions, or some parameters can more naturally be grouped in a
class.

IIRC Han-Wen suggested to write Python scripts, because the mess of
make and shell got unmaintainable, so adding to that would be unwise.
Using python should result in a reduction of lines of code, and much
clearer lines as well.  I'm not sure that was achieved yet.  Comparing
hardlink_tree and say this snippet

-   for dir in $(outdir)/web-root/ ; do  \
-   cd $$dir  \
-   for a in `find . -name out-www`; do \
-   rsync -a  --link-dest $$a/ $$a/ $$a/.. ; \
-   rm -rf $$a ; \
-   done \
-   done

(hardlink_tree does much more?).  Having all logic in python is VERY
NICE, but I would not hesitate to call system ('rsync ..') from python
if that would dramatically simplify things.

Jan.

-- 
Jan Nieuwenhuizen [EMAIL PROTECTED] | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien   | http://www.lilypond.org


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel