Hi, believe it or not, but I've found some time to dig into the testing stuff of perl. So (finally) I've written some test cases and documentation for the <lang> tag together with the underlaying multilanguage support of webmake.
Thanks, Jason, for incorporating my patches despite my unfulfilled promises of sending you somehting like this earlier. Chris -- FB6A 3472 A931 5FA1 8597 F04E 6764 AF48 F863 1501
diff -Nur HTML-WebMake-2.4-orig/doc/multilang.txt HTML-WebMake-2.4/doc/multilang.txt --- HTML-WebMake-2.4-orig/doc/multilang.txt Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/doc/multilang.txt Tue Jul 29 23:09:30 2003 @@ -0,0 +1,224 @@ +<wmmeta name="Title" value="Multi language support and the <lang> Tag" /> +<wmmeta name="Section" value="05-meta" /> +<wmmeta name="Score" value="55" /> +<wmmeta name="Abstract"> +Support for multilingual web pages from single source files. +</wmmeta> + +All meta tags can be augmented by an optional ##lang="<languageid>"## attribute +which associates the meta content with a certain language. Together with the +##lang## HTML-tag (which is included via ##<use plugin="lang_tag.wmk">## within +the webmake file), this allows for multi language contents with different meta +contents for different languages. + + +How to define language dependent meta data +------------------------------------------ + +Let's start with an example: + +<safe> + <content name="foo"> + < wmmeta lang="en" name="Title" value="English Title" /> + < wmmeta lang="de" name="Title" value="Deutscher Titel" /> + < wmmeta name="Section" value="section" /> + < wmmeta name="Score" value="10" /> + < wmmeta lang="en" name="Abstract"> + This is an english abstract + </wmmeta> + < wmmeta lang="de" name="Abstract"> + Das ist eine deutsche Zusammenfassung + </wmmeta> + + Foo foo foo foo bar. etc. + </content> +</safe> + + +We've set two language dependent metadata item (##Title## and ##Abstract##) +by using the ##lang## attribute and two others (##Section## and ##Score##) which +lack this attribute. + +On producing the output, metadata item without a language attributes are used +as __default__ values if no appropriate metadata item for the currently +selected language can be found. If you've more than two language, this makes it +possible to define a default value for all languages but redefine it for only +one special langague. + + +It is also possible to define language dependet metadata items if you use +##metatables##: + + +<safe> + <metatable delimiter="|"> + .|up|score + content.txt|index.txt|70 + .|title|abstract + ..|lang|de + content.txt|Bildergalerie|Einige Bildergalerien. + ..|lang|en + content.txt|Galleries|Some image galleries. + </metatable> +</safe> + +Here ##up## and ##score## is the same for all languages for ##content.txt## and +##title## and ##abstract## are set differently for english and german. The +general format for the metatable language support is: + +<safe> + ..<delimiter>lang<delimter><languageid> +</safe> + +The two points start a __sublevel__ within the metadata item list defined +in the line starting with one point. The next entry defines the feature to set, +in this case ##lang## (which is the only one supported up to now). And the +third entry defines the value for the feature (here the language identifier). + + +Remark +====== + +Other sources of meta data are **not** supported, up to now. This includes: + + - self defined ##metaset##s + - metadata defined in xml files (not csv) + + + +How to set language dependent content parts +-------------------------------------------- + +The language dependent content is set by using the ##lang## HTML tag. + +<safe> + <lang id="de"> + Das hier erscheint nur auf der deutschen Seite. + </lang> + <lang id="en"> + This appears only on the english page. + </lang> + + <img src="nice_picture.jpg" /> <!-- this appears always --> +</safe> + +How to output proper html files +-------------------------------------------- + +On outputting the html pages the attribute ##lang## has to be define to select +a certain language. Usually one defines a content containing the selected language +so it can be used within the content files. + + +Here's an example: + +<safe> + <!-- output english pages --> + <template name="curlang">en</template> + <attrdefault name="lang" value="${curlang}" > + + <out name="index" file="index.html.${curlang}"> + ${header} + ${index_text} + ${footer} + </out> + + </attrdefault> + + <!-- output german pages --> + <template name="curlang">de</template> + <attrdefault name="lang" value="${curlang}" > + + <out name="index" file="index.html.${curlang}"> + ${header} + ${index_text} + ${footer} + </out> + + + </attrdefault> + </usemetaset> + +</safe> + + +The ##<out ...>## part can be sourced out into a file on its own which is +included instead of rewriting the code for each language section. + + +Further Information and some Tricks +---------------------------------- + +The language support is implemented by adding ##::<languageid>## to the keys +used to store all meta information and urls together with the content data. +Knowing this, it is possible to manipulate all language dependent entries +directly. + +The following sections give some examples of how to make use of this. + + +Referring to output files of another language +=========================================== + +In order to be able to switch languages one has to use urls which refer to +output pages of another language. This is like avoiding the automatic language +detection feature while using ##$(url)## and indicating directly to which +language file to use. + +The following example shows a code snippet from a page header template which +makes it possible to switch from the englisch to the german representation of +one page and visaverse. + +<safe> + <lang id=de> + <a href="$(${WebMake.OutName}::en)"> + <img src="$(englishFlag.gif)" alt="EN"> + </a> + </lang> + <lang id=en> + <a href="$(${WebMake.OutName}::de)"> + <img src="$(germanFlag.gif)" alt="DE"> + </a> + </lang> +</safe> + + + + +Embedding CGI scripts with language support +=========================================== + +In this example the urls for a external cgi script, for which display templates +are generated with webmake, are set from a small perl script: + + +<safe> + <attrdefault name="map" value="true" > + <media src="gallery" name="gallery.php3" /> + </attrdefault> + + <metatable delimiter="|"> + .|up|score + gallery.php3|index.txt|70 + .|title|abstract + ..|lang|de + gallery.php3|Bildergalerie|Alle möglichen Bildergalerien + ..|lang|en + gallery.php3|Galleries|All different kinds of image galleries + </metatable> + + <!-- add language url's, not nice but it works --> + <{perl + my $base = get_url("gallery.php3"); + set_url("gallery.php3::de", $base . "?lang=de"); + set_url("gallery.php3::en", $base . "?lang=en"); + ''; + }> + +</safe> + + +After this, ##$(gallery.php3)## within a content gives +##<path_to_script>/gallery.php3?lang=de## with language set to ##de## and +##<path_to_script>/gallery.php3?lang=en## with language set to ##en##. + diff -Nur HTML-WebMake-2.4-orig/lib/HTML/WebMake/NormalContent.pm HTML-WebMake-2.4/lib/HTML/WebMake/NormalContent.pm --- HTML-WebMake-2.4-orig/lib/HTML/WebMake/NormalContent.pm Sun Dec 9 11:07:57 2001 +++ HTML-WebMake-2.4/lib/HTML/WebMake/NormalContent.pm Sun Jul 27 14:46:12 2003 @@ -663,10 +663,19 @@ return if ($filename =~ /^\(/); # (eval), (dep_ignore) etc. - if (!$self->{no_map} && !defined $self->{reffed_in_url}) { + my $main = $self->{main}; + my $l = $self->{main}->{current_subst}->{lang}; + my $lang_key = ""; + my $reffed_key = "reffed_in_url"; + + $lang_key = "::".$l if defined($l); + $reffed_key .= "::".$l if defined($l); + + + if (!$self->{no_map} && !defined $self->{$reffed_key}) { dbg2 ($self->as_string().": add ref from url $filename"); - $self->{reffed_in_url} = $filename; - $self->{main}->getcache()->put_metadata ($self->{name}.".url", $filename); + $self->{$reffed_key} = $filename; + $self->{main}->getcache()->put_metadata ($self->{name}.$lang_key.".url", $filename); } } @@ -678,13 +687,24 @@ return ''; } - my $url = $self->{reffed_in_url}; + my $main = $self->{main}; + my $l = $self->{main}->{current_subst}->{lang}; + my $lang_key = ""; + my $reffed_key = "reffed_in_url"; + + $lang_key = "::".$l if defined($l); + $reffed_key .= "::".$l if defined($l); + + my $url = $self->{$reffed_key}; + + dbg2("NormalContent->get_url: got \"".$url."\"") if defined $url; if (defined $url) { return $url; } - $url = $self->{main}->getcache()->get_metadata ($self->{name}.".url"); + dbg2("NormalContent->get_url: try to get \"".$self->{name}.$lang_key.".url\""); + $url = $self->{main}->getcache()->get_metadata ($self->{name}.$lang_key.".url"); if (defined $url) { - $self->{reffed_in_url} = $url; + $self->{$reffed_key} = $url; return $url; } diff -Nur HTML-WebMake-2.4-orig/lib/HTML/WebMake/PerlLib/lang_tag.wmk HTML-WebMake-2.4/lib/HTML/WebMake/PerlLib/lang_tag.wmk --- HTML-WebMake-2.4-orig/lib/HTML/WebMake/PerlLib/lang_tag.wmk Thu Nov 22 10:38:33 2001 +++ HTML-WebMake-2.4/lib/HTML/WebMake/PerlLib/lang_tag.wmk Sun Jul 27 14:45:09 2003 @@ -1,3 +1,45 @@ +<!-- +=head1 NAME + +lang_tag.wmk - markup text for language dependent output + +=head1 LOADING + + <use plugin="lang_tag" /> + +=head1 HTML TAGS + + < lang id=... > + + < /lang > + +=head1 DESCRIPTION + +This WebMake Perl library provides a support tag for writing multi language +input files. + +During the output stage the C<id> is compared to attribute C<lang> and only on +equality the text is included into the output. +Text parts which are not included int C<lang> tags are always included, regardless +of the C<lang>-attribute. + +This makes it possible to have one file which contains the content for several +languages and common parts have to be written only once + +For more information on the multi language support please consult the C<webmake> +documenation. + +=head1 ATTRIBUTES + +=over 4 + +=item id + +This is the language identifier for the text section. + +=cut + +--> <{perl $self->define_tag ("lang", \&handle_lang_tag, qw(id)); diff -Nur HTML-WebMake-2.4-orig/t/data/multilangmeta.data/bar.txt HTML-WebMake-2.4/t/data/multilangmeta.data/bar.txt --- HTML-WebMake-2.4-orig/t/data/multilangmeta.data/bar.txt Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/data/multilangmeta.data/bar.txt Mon Jul 28 20:42:57 2003 @@ -0,0 +1,7 @@ +<wmmeta lang="en" name="title" value="This is bar." /> +<wmmeta lang="de" name="title" value="Das ist bar." /> +<wmmeta name="title" value="Default bar title." /> +<wmmeta name="score" value=20 /> + +This is the bar document. The title looks like this: $[this.title] + diff -Nur HTML-WebMake-2.4-orig/t/data/multilangmeta.data/foo.txt HTML-WebMake-2.4/t/data/multilangmeta.data/foo.txt --- HTML-WebMake-2.4-orig/t/data/multilangmeta.data/foo.txt Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/data/multilangmeta.data/foo.txt Mon Jul 28 20:42:45 2003 @@ -0,0 +1,6 @@ +<wmmeta lang="en" name="title" value="This is foo." /> +<wmmeta lang="de" name="title" value="Das ist foo." /> +<wmmeta name="title" value="Default foo title." /> +<wmmeta name="score" value=10 /> + +This is the foo document. The title looks like this: $[this.title] diff -Nur HTML-WebMake-2.4-orig/t/data/multilangmeta.wmk HTML-WebMake-2.4/t/data/multilangmeta.wmk --- HTML-WebMake-2.4-orig/t/data/multilangmeta.wmk Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/data/multilangmeta.wmk Mon Jul 28 20:46:27 2003 @@ -0,0 +1,59 @@ + +<contents src="data/multilangmeta.data" name="*.txt" format="text/et" /> + +<content name=index format="text/et"> + +Title for foo: ''$[foo.txt.title]'' Foo's score: $[foo.txt.score] +Link: <a href=$(foo)>here</a> + +Title for bar: ''$[bar.txt.title]'' Bar's score: $[bar.txt.score] +Link: <a href=$(bar)>here</a> + +</content> + +<content name=header format="text/et"> +$[this.title] +------------- +</content> + + +<attrdefault name="lang" value="de" > +<out file=log/multilangmeta_de.html>${index}</out> + +<out name=foo file=log/multilangmeta_foo_de.html> +${header} +Title in out item: $[this.title] +${foo.txt}</out> + +<out name=bar file=log/multilangmeta_bar_de.html> +${header} +Title in out item: $[this.title] +${bar.txt}</out> +</attrdefault> + +<attrdefault name="lang" value="en" > +<out file=log/multilangmeta_en.html>${index}</out> + +<out name=foo file=log/multilangmeta_foo_en.html> +${header} +Title in out item: $[this.title] +${foo.txt}</out> + +<out name=bar file=log/multilangmeta_bar_en.html> +${header} +Title in out item: $[this.title] +${bar.txt}</out> +</attrdefault> + + +<out file=log/multilangmeta.html>${index}</out> + +<out name=foo file=log/multilangmeta_foo.html> +${header} +Title in out item: $[this.title] +${foo.txt}</out> + +<out name=bar file=log/multilangmeta_bar.html> +${header} +Title in out item: $[this.title] +${bar.txt}</out> diff -Nur HTML-WebMake-2.4-orig/t/data/multilangmetatable.wmk HTML-WebMake-2.4/t/data/multilangmetatable.wmk --- HTML-WebMake-2.4-orig/t/data/multilangmetatable.wmk Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/data/multilangmetatable.wmk Tue Jul 29 20:29:43 2003 @@ -0,0 +1,80 @@ +<webmake> + +<media src="data/contentsfind.data" name=".../*.txt" /> + +<metatable> +. section score +foo.txt sec1 50 +dir1/bar.txt sec2 30 +dir2/dir2a/baz.txt sec1 20 +. title +foo.txt Default foo +dir1/bar.txt Default bar +dir2/dir2a/baz.txt Default baz +.. lang de +foo.txt Das ist foo +dir1/bar.txt Das ist bar +dir2/dir2a/baz.txt Das ist baz +.. lang en +foo.txt This is foo +dir1/bar.txt This is bar +dir2/dir2a/baz.txt This is baz +</metatable> + +<attrdefault name="lang" value="en" > +<out file=log/multilangmetatable.html> +Name: foo +Title: $[foo.txt.title] +Section: $[foo.txt.section] +Score: $[foo.txt.score] + +Name: dir1/bar +Title: $[dir1/bar.txt.title] +Section: $[dir1/bar.txt.section] +Score: $[dir1/bar.txt.score] + +Name: dir2/dir2a/baz +Title: $[dir2/dir2a/baz.txt.title] +Section: $[dir2/dir2a/baz.txt.section] +Score: $[dir2/dir2a/baz.txt.score] +</out> +</attrdefault> + +<attrdefault name="lang" value="de" > +<out file=log/multilangmetatable_de.html> +Name: foo +Title: $[foo.txt.title] +Section: $[foo.txt.section] +Score: $[foo.txt.score] + +Name: dir1/bar +Title: $[dir1/bar.txt.title] +Section: $[dir1/bar.txt.section] +Score: $[dir1/bar.txt.score] + +Name: dir2/dir2a/baz +Title: $[dir2/dir2a/baz.txt.title] +Section: $[dir2/dir2a/baz.txt.section] +Score: $[dir2/dir2a/baz.txt.score] +</out> +</attrdefault> + +<attrdefault name="lang" value="fr" > +<out file=log/multilangmetatable_fr.html> +Name: foo +Title: $[foo.txt.title] +Section: $[foo.txt.section] +Score: $[foo.txt.score] + +Name: dir1/bar +Title: $[dir1/bar.txt.title] +Section: $[dir1/bar.txt.section] +Score: $[dir1/bar.txt.score] + +Name: dir2/dir2a/baz +Title: $[dir2/dir2a/baz.txt.title] +Section: $[dir2/dir2a/baz.txt.section] +Score: $[dir2/dir2a/baz.txt.score] +</out> +</attrdefault> + diff -Nur HTML-WebMake-2.4-orig/t/multilangmeta.t HTML-WebMake-2.4/t/multilangmeta.t --- HTML-WebMake-2.4-orig/t/multilangmeta.t Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/multilangmeta.t Tue Jul 29 00:27:28 2003 @@ -0,0 +1,82 @@ +#!/usr/bin/perl -w + +use lib '.'; use lib 't'; +use WMTest; webmake_t_init("multilangmeta"); +use Test; BEGIN { plan tests => 37 }; + +clear_cache_dir(); + +# --------------------------------------------------------------------------- + +%patterns = ( + + q{Title for foo: "Default foo title." Foo's score: 10}, + 'index_title_foo', + + q{Title for bar: "Default bar title." Bar's score: 20}, + 'index_title_bar', + + q{Link: <a href="../log/multilangmeta_foo.html">here</a>}, + 'index_link_foo', + + q{Link: <a href="../log/multilangmeta_bar.html">here</a>}, + 'index_link_bar', + + q{Title in out item: Default foo title.}, + 'title_file_foo', + + q{Title in out item: Default bar title.}, + 'title_file_bar', + + q{Title for foo: "Das ist foo." Foo's score: 10}, + 'index_title_score_foo_de', + + q{Title for bar: "Das ist bar." Bar's score: 20}, + 'index_title_score_bar_de', + + q{Link: <a href="../log/multilangmeta_foo_de.html">here</a>}, + 'index_link_foo_de', + + q{Link: <a href="../log/multilangmeta_bar_de.html">here</a>}, + 'index_link_bar_de', + + q{Title in out item: Das ist foo.}, + 'foo_title_de', + + q{Title in out item: Das ist bar.}, + 'bar_title_de', + + q{Title for foo: "This is foo." Foo's score: 10}, + 'index_title_score_foo_en', + + q{Title for bar: "This is bar." Bar's score: 20}, + 'index_title_score_bar_en', + + q{Link: <a href="../log/multilangmeta_foo_en.html">here</a>}, + 'index_link_foo_en', + + q{Link: <a href="../log/multilangmeta_bar_en.html">here</a>}, + 'index_link_bar_en', + + q{Title in out item: This is foo.}, + 'foo_title_en', + + q{Title in out item: This is bar.}, + 'bar_title_en' + +); + +# --------------------------------------------------------------------------- + +ok (wmrun ("-F -f data/$testname.wmk", \&patterns_run_cb)); +checkfile ($testname."_foo.html", \&patterns_run_cb); +checkfile ($testname."_bar.html", \&patterns_run_cb); +checkfile ($testname."_de.html", \&patterns_run_cb); +checkfile ($testname."_foo_de.html", \&patterns_run_cb); +checkfile ($testname."_bar_de.html", \&patterns_run_cb); +checkfile ($testname."_en.html", \&patterns_run_cb); +checkfile ($testname."_foo_en.html", \&patterns_run_cb); +checkfile ($testname."_bar_en.html", \&patterns_run_cb); +ok_all_patterns(); + + diff -Nur HTML-WebMake-2.4-orig/t/multilangmetaset.t HTML-WebMake-2.4/t/multilangmetaset.t --- HTML-WebMake-2.4-orig/t/multilangmetaset.t Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/multilangmetaset.t Tue Jul 29 23:02:50 2003 @@ -0,0 +1,72 @@ +#!/usr/bin/perl -w + +use lib '.'; use lib 't'; +use WMTest; webmake_t_init("multilangmetaset"); +use Test; BEGIN { plan tests => 9 }; + +# --------------------------------------------------------------------------- + +$file = q{ +<webmake> + <metaset id="test"> + T_i_t_l_e_: type=string default="Default title." + _S_c_o_r_e: type=numeric default="20" + </metaset> + + <usemetaset id="test"> + <content name=foo format="text/et"> + <wmmeta name="_S_c_o_r_e" value=10 /> + <wmmeta lang="de" name="T_i_t_l_e_" value="Das ist foo" /> + + + This is the foo document. + The title looks like this: $[this.T_i_t_l_e_] + The score looks like this: $[this._S_c_o_r_e] + </content> + + <content name=bar format="text/et"> + <wmmeta name="T_i_t_l_e_" value="Default bar." /> + <wmmeta lang="de" name="T_i_t_l_e_" value="Das ist bar" /> + + This is the bar document. + The title looks like this: $[this.T_i_t_l_e_] + The score looks like this: $[this._S_c_o_r_e] + </content> + + <content name=index format="text/et"> + + Title for foo: ''$[foo.T_i_t_l_e_]'' Foo's score: $[foo._S_c_o_r_e] + + Title for bar: ''$[bar.T_i_t_l_e_]'' Bar's score: $[bar._S_c_o_r_e] + + </content> + + + <attrdefault name="lang" value="en" > + <out file=log/multilangmetaset.html>${index}${foo}${bar}</out> + </attrdefault> + <attrdefault name="lang" value="de" > + <out file=log/multilangmetaset_de.html>${index}${foo}${bar}</out> + </attrdefault> + </usemetaset> +</webmake> +}; + +%patterns = ( + q{Title for foo: "This is foo." Foo's score: 10}, 'fooidx', + + q{Title for bar: "This is bar." Bar's score: 20}, 'baridx', + + q{This is the foo document. The title looks like this: 20}, 'foodoc', + + q{This is the bar document. The title looks like this: This is foo.}, + 'bardoc', + +); + +# --------------------------------------------------------------------------- + +wmfile ($file); +ok (wmrun ("-F -f log/test.wmk", \&patterns_run_cb)); +ok_all_patterns(); + diff -Nur HTML-WebMake-2.4-orig/t/multilangmetatable.t HTML-WebMake-2.4/t/multilangmetatable.t --- HTML-WebMake-2.4-orig/t/multilangmetatable.t Thu Jan 1 01:00:00 1970 +++ HTML-WebMake-2.4/t/multilangmetatable.t Tue Jul 29 20:30:47 2003 @@ -0,0 +1,70 @@ +#!/usr/bin/perl -w + +use lib '.'; use lib 't'; +use WMTest; webmake_t_init("multilangmetatable"); +use Test; BEGIN { plan tests => 19 }; + +# --------------------------------------------------------------------------- + +%patterns = ( + q{Name: foo + Title: This is foo + Section: sec1 + Score: 50}, + 'foo_en', + + q{Name: dir1/bar + Title: This is bar + Section: sec2 + Score: 30}, + 'bar_en', + + q{Name: dir2/dir2a/baz + Title: This is baz + Section: sec1 + Score: 20}, + 'baz_en', + + q{Name: foo + Title: Das ist foo + Section: sec1 + Score: 50}, + 'foo_de', + + q{Name: dir1/bar + Title: Das ist bar + Section: sec2 + Score: 30}, + 'bar_de', + + q{Name: dir2/dir2a/baz + Title: Das ist baz + Section: sec1 + Score: 20}, + 'baz_de', + + q{Name: foo + Title: Default foo + Section: sec1 + Score: 50}, + 'foo_fr', + + q{Name: dir1/bar + Title: Default bar + Section: sec2 + Score: 30}, + 'bar_fr', + + q{Name: dir2/dir2a/baz + Title: Default baz + Section: sec1 + Score: 20}, + 'baz_fr', + + +); + +ok (wmrun ("-F -f data/$testname.wmk", \&patterns_run_cb)); +checkfile ($testname."_de.html", \&patterns_run_cb); +checkfile ($testname."_fr.html", \&patterns_run_cb); +ok_all_patterns();