Re: Do we support user defined layout?
Georg Baum wrote: Abdelrazak Younes wrote: My idea would be to cache the translated text instead of translating each time updateLabels() is called. But I am not sure I can do that on untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. But we have to reconfigure lyx when a new layout is added? So if this cach is built at least at reconfigure time, then it won't ever be out of date. Or it could be built when lyx starts, or when the document loads - whatever seems convenient. Helge Hafting
Re: Do we support user defined layout?
Georg Baum wrote: Abdelrazak Younes wrote: My idea would be to cache the translated text instead of translating each time updateLabels() is called. But I am not sure I can do that on untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. But we have to "reconfigure" lyx when a new layout is added? So if this cach is built at least at reconfigure time, then it won't ever be out of date. Or it could be built when lyx starts, or when the document loads - whatever seems convenient. Helge Hafting
Re: Do we support user defined layout?
[EMAIL PROTECTED] writes: | On Fri, 12 Jan 2007, Georg Baum wrote: | | Abdelrazak Younes wrote: | | Georg Baum wrote: | They are not different than the translatable ones. The only problem I can | imagine is that the cache could get out of date if somebody changes the | layout file. | | Do you mean in the Document settings? If yes, then we just need to reset | the cache when this is changed. | | No, I mean if somebody changes the layout file on disk. I don't know | whether we read layout files only on startup, or always if a buffer | is loaded, so I don't know whether this is already a problem or not. | | When I messed with layouts as a user, LyX only read layout files on | startup. So that's how the older versions did it and AFAIK this has | not changed. However, I really hope LyX will be able to to re-read the | layout files in the future so I think it'd be bad to add a constraint. We read layout files lazily, they are read when first used. What do we gain by re-reading? -- Lgb
Re: Do we support user defined layout?
[EMAIL PROTECTED] writes: | On Fri, 12 Jan 2007, Georg Baum wrote: | | > Abdelrazak Younes wrote: | > | >> Georg Baum wrote: | >>> They are not different than the translatable ones. The only problem I can | >>> imagine is that the cache could get out of date if somebody changes the | >>> layout file. | >> | >> Do you mean in the Document settings? If yes, then we just need to reset | >> the cache when this is changed. | > | > No, I mean if somebody changes the layout file on disk. I don't know | > whether we read layout files only on startup, or always if a buffer | > is loaded, so I don't know whether this is already a problem or not. | | When I messed with layouts as a user, LyX only read layout files on | startup. So that's how the older versions did it and AFAIK this has | not changed. However, I really hope LyX will be able to to re-read the | layout files in the future so I think it'd be bad to add a constraint. We read layout files lazily, they are read when first used. What do we gain by re-reading? -- Lgb
Do we support user defined layout?
Hello,
I finally know what is causing the slowness in updateLabel() on Windows!
The culprit is the translation done in Buffer::translateLabel():
docstring const Buffer::translateLabel(docstring const label) const
{
if (support::isAscii(label))
// Probably standard layout, try to translate
return B_(to_ascii(label));
else
// This must be a user defined layout. We cannot translate
// this, since gettext accepts only ascii keys.
return label;
}
If I short-cuts this and return label (thus avoiding the gettext
translation), the paragraph breaking is very fast!
My idea would be to cache the translated text instead of translating
each time updateLabels() is called. But I am not sure I can do that on
untranslatable layouts.
I seem to remember from previous discussions that user-defined layout
command are not allowed any more.
So question to Georg, Jose, JMarc or ???:
How is this layout business related to that translateLabel() method? Can
I just ditch the method and use _() directly?
Another idea would be to translate the layout on first loading and use
that later on. Then we could continue to support untranslatable layouts.
I am not sure I got everything right so please explain me.
Abdel.
Re: Do we support user defined layout?
Abdelrazak Younes wrote: My idea would be to cache the translated text instead of translating each time updateLabels() is called. But I am not sure I can do that on untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. I seem to remember from previous discussions that user-defined layout command are not allowed any more. Wrong. That would mean that nobody could create custom layouts in ~/.lib/layouts. There is no reason to forbid that. How is this layout business related to that translateLabel() method? Can I just ditch the method and use _() directly? No. _() would assert if a user defined layout contained a label name with non-ascii characters. That was the very reason to introduce Buffer::translateLabel. Besides the slowness is in _(), not the additional ascii test. Another idea would be to translate the layout on first loading and use that later on. Then we could continue to support untranslatable layouts. That would be problematic since the translation depends on the buffer language. But if you introduce a cache at buffer level it should work. I am not sure I got everything right so please explain me. I hope it is clear now. Georg
Re: Do we support user defined layout?
Georg Baum wrote: Abdelrazak Younes wrote: I seem to remember from previous discussions that user-defined layout command are not allowed any more. Wrong. That would mean that nobody could create custom layouts in ~/.lib/layouts. There is no reason to forbid that. Now I see that I overread 'command'. What you mean are style names, but what is translated here are static labels. Georg
Re: Do we support user defined layout?
Georg Baum wrote:
Georg Baum wrote:
Abdelrazak Younes wrote:
I seem to remember from previous discussions that user-defined layout
command are not allowed any more.
Wrong. That would mean that nobody could create custom layouts in
~/.lib/layouts. There is no reason to forbid that.
Now I see that I overread 'command'. What you mean are style names, but what
is translated here are static labels.
Yes, that's what I meant. The problem is that I don't know what's there
to translate. In a koma-script or article layout, here is what
layout-labelstring() gives inside 'expandLabel()':
raw fmt: \arabic{chapter}.\arabic{section}
raw fmt: @[EMAIL PROTECTED]
Am I right here that there is nothing to translate?
For the 'report' layout' we can have something like this:
raw fmt: Chapter \arabic{chapter}
raw fmt: \arabic{chapter}.\arabic{section}
Here, is it right that the only translatable part is the word Chapter?
If yes, wouldn't it be possible to translate that word once and for all
at layout loading? Then we could avoid translateLabel() altogether.
Abdel.
Re: Do we support user defined layout?
Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes: Abdelrazak My idea would be to cache the translated text instead of Abdelrazak translating each time updateLabels() is called. But I am Abdelrazak not sure I can do that on untranslatable layouts. This is a problem with multiple languages. I _think_ the problem is that switching language is expensive with gettext (I am not sure gettext is designed to change languages very often, like we do). If this is the case, then we could cache string/lang pairs. JMarc
Re: Do we support user defined layout?
Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:
Abdelrazak Yes, that's what I meant. The problem is that I don't know
Abdelrazak what's there to translate. In a koma-script or article
Abdelrazak layout, here is what
layout- labelstring() gives inside 'expandLabel()':
Abdelrazak raw fmt: \arabic{chapter}.\arabic{section} raw fmt:
Abdelrazak @[EMAIL PROTECTED]
Abdelrazak Am I right here that there is nothing to translate?
No. Look for example at this eu.po snippet:
msgid Chapter \\arabic{chapter}
msgstr \\arabic{chapter}. kapitulua
Basque does not have the same conventions as we do. These are rare
cases, but they exist.
JMarc
Re: Do we support user defined layout?
Jean-Marc Lasgouttes wrote:
Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:
Abdelrazak Yes, that's what I meant. The problem is that I don't know
Abdelrazak what's there to translate. In a koma-script or article
Abdelrazak layout, here is what
layout- labelstring() gives inside 'expandLabel()':
Abdelrazak raw fmt: \arabic{chapter}.\arabic{section} raw fmt:
Abdelrazak @[EMAIL PROTECTED]
Abdelrazak Am I right here that there is nothing to translate?
No. Look for example at this eu.po snippet:
msgid Chapter \\arabic{chapter}
msgstr \\arabic{chapter}. kapitulua
Basque does not have the same conventions as we do. These are rare
cases, but they exist.
Oh... shit.
Abdel.
Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote:
Abdelrazak Younes wrote:
Jean-Marc Lasgouttes wrote:
Abdelrazak == Abdelrazak Younes
[EMAIL PROTECTED] writes:
Abdelrazak My idea would be to cache the translated text instead of
Abdelrazak translating each time updateLabels() is called. But I am
Abdelrazak not sure I can do that on untranslatable layouts.
This is a problem with multiple languages. I _think_ the problem is
that switching language is expensive with gettext (I am not sure
gettext is designed to change languages very often, like we do).
I think you're right.
If this is the case, then we could cache string/lang pairs.
You mean at gettext.[Ch] level?
Or rather at messages.[Ch] level. As this is set to one unique language
Apparently, this is not true and I don't understand why. The language is
set at construction of the Message class. A language change in the
document will reset the Message object to the new language. Up to that
point I understand.
But, updateLabel() is using Messages::Pimpl::get() and this can change
the language at each call! This is because of this code that checks ans
sets environment variables:
docstring const get(string const m) const
{
if (m.empty())
return from_ascii(m);
// In this order, see support/filetools.C:
string lang = getEnv(LC_ALL);
if (lang.empty()) {
lang = getEnv(LC_MESSAGES);
if (lang.empty()) {
lang = getEnv(LANG);
if (lang.empty())
lang = C;
}
}
#ifdef HAVE_LC_MESSAGES
char const * lc_msgs = setlocale(LC_MESSAGES, lang_.c_str());
#endif
This last line in particular looks suspicious. Why do we set environment
variable at each translation request?
This other chunk of code rings a bell in the context of the hebrew RTL
bug (related to LC_TYPE):
// CTYPE controls what getmessage thinks what encoding the po file uses
char const * lc_ctype = setlocale(LC_CTYPE, NULL);
string oldCTYPE = lc_ctype ? lc_ctype : ;
I don't know anything about gettext but this stuff looks awfully
complicated. Couldn't we simplify the thing a bit?
Abdel.
Re: Do we support user defined layout?
Georg Baum wrote: Abdelrazak Younes wrote: My idea would be to cache the translated text instead of translating each time updateLabels() is called. But I am not sure I can do that on untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. Do you mean in the Document settings? If yes, then we just need to reset the cache when this is changed. Another idea would be to translate the layout on first loading and use that later on. Then we could continue to support untranslatable layouts. That would be problematic since the translation depends on the buffer language. But if you introduce a cache at buffer level it should work. Problem is that a single Buffer can presents multiple languages as explained in the other part of the thread. I am not sure I got everything right so please explain me. I hope it is clear now. Clearer yes, thanks. Abdel.
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote: This last line in particular looks suspicious. Why do we set environment variable at each translation request? Because the environment variable determines what language gettext will use, and we don't know what other message was translated before. To my knowledge no library call of gettext exists that takes the language as a function argument rather than an environment variable. This other chunk of code rings a bell in the context of the hebrew RTL bug (related to LC_TYPE): // CTYPE controls what getmessage thinks what encoding the po file uses char const * lc_ctype = setlocale(LC_CTYPE, NULL); string oldCTYPE = lc_ctype ? lc_ctype : ; But setting LC_TYPE is needed for gettext. I don't know anything about gettext but this stuff looks awfully complicated. Couldn't we simplify the thing a bit? I don't think so. This awfully complicated stuff is the result of many bug fixes and a lot of work of several people. It is so complicated because some gettext versions are broken (especially on windows). I once tried to simplify this but only made Michael Gerz angry who suddenly discovered lots of bugs. Unless you really want to spend a lot of time yo learn all aspects about gettext I suggest that you leave this alone. Georg
Re: Do we support user defined layout?
Abdelrazak Younes wrote: Georg Baum wrote: They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. Do you mean in the Document settings? If yes, then we just need to reset the cache when this is changed. No, I mean if somebody changes the layout file on disk. I don't know whether we read layout files only on startup, or always if a buffer is loaded, so I don't know whether this is already a problem or not. Problem is that a single Buffer can presents multiple languages as explained in the other part of the thread. Indeed. Then maybe a cache per layout but with different languages would work. Georg
Re: Translation madness! (was Re: Do we support user defined layout?)
Georg Baum wrote: Abdelrazak Younes wrote: This last line in particular looks suspicious. Why do we set environment variable at each translation request? Because the environment variable determines what language gettext will use, and we don't know what other message was translated before. To my knowledge no library call of gettext exists that takes the language as a function argument rather than an environment variable. [...] Unless you really want to spend a lot of time yo learn all aspects about gettext I suggest that you leave this alone. OK, thanks for the explanation. I am not pretending that I will solve everything, don't be afraid ;-). We do have an internal in source gettext. Couldn't we just support that and live all other version aside? Maybe this would simplify the code a bit, dunno. One other thing that may be changed also is the way we load po files. It seems that there is some caching done inside gettext but it is disabled as soon as we change the po file. This is explained here: http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152 Maybe there's something wrong in the way we load the po files? Just asking to you knowledgeable people... Abdel.
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote:
Georg Baum wrote:
Abdelrazak Younes wrote:
This last line in particular looks suspicious. Why do we set environment
variable at each translation request?
Because the environment variable determines what language gettext will
use,
and we don't know what other message was translated before. To my
knowledge
no library call of gettext exists that takes the language as a function
argument rather than an environment variable.
[...]
Unless you really want to spend a lot of time yo learn all aspects about
gettext I suggest that you leave this alone.
OK, thanks for the explanation. I am not pretending that I will solve
everything, don't be afraid ;-). We do have an internal in source
gettext. Couldn't we just support that and live all other version aside?
Maybe this would simplify the code a bit, dunno.
One other thing that may be changed also is the way we load po files. It
seems that there is some caching done inside gettext but it is disabled
as soon as we change the po file. This is explained here:
http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152
Maybe there's something wrong in the way we load the po files? Just
asking to you knowledgeable people...
At this link they say something interesting WRT changing language:
http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC155
I don't see any use of this '_nl_msg_cat_cntr' variable in the code.
Maybe that's a solution. Did you tried that in the past?
10.5 Being a gettext grok
To fully exploit the functionality of the GNU gettext library it is
surely helpful to read the source code. But for those who don't want to
spend that much time in reading the (sometimes complicated) code here is
a list comments:
* Changing the language at runtime
For interactive programs it might be useful to offer a selection
of the used language at runtime. To understand how to do this one need
to know how the used language is determined while executing the gettext
function. The method which is presented here only works correctly with
the GNU implementation of the gettext functions.
In the function dcgettext at every call the current setting of
the highest priority environment variable is determined and used.
Highest priority means here the following list with decreasing priority:
1. LANGUAGE
2. LC_ALL
3. LC_xxx, according to selected locale
4. LANG
Afterwards the path is constructed using the found value and the
translation file is loaded if available.
What is now when the value for, say, LANGUAGE changes. According
to the process explained above the new value of this variable is found
as soon as the dcgettext function is called. But this also means the
(perhaps) different message catalog file is loaded. In other words: the
used language is changed.
But there is one little hook. The code for gcc-2.7.0 and up provides
some optimization. This optimization normally prevents the calling of
the dcgettext function as long as no new catalog is loaded. But if
dcgettext is not called the program also cannot find the LANGUAGE
variable be changed (see section 10.2.7 Optimization of the *gettext
functions). A solution for this is very easy. Include the following code
in the language switching function.
/* Change language. */
setenv (LANGUAGE, fr, 1);
/* Make change known. */
{
extern int _nl_msg_cat_cntr;
++_nl_msg_cat_cntr;
}
Abdel.
Re: Do we support user defined layout?
On Fri, 12 Jan 2007, Georg Baum wrote: Abdelrazak Younes wrote: Georg Baum wrote: They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. Do you mean in the Document settings? If yes, then we just need to reset the cache when this is changed. No, I mean if somebody changes the layout file on disk. I don't know whether we read layout files only on startup, or always if a buffer is loaded, so I don't know whether this is already a problem or not. When I messed with layouts as a user, LyX only read layout files on startup. So that's how the older versions did it and AFAIK this has not changed. However, I really hope LyX will be able to to re-read the layout files in the future so I think it'd be bad to add a constraint. cheers /Christian -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote: OK, thanks for the explanation. I am not pretending that I will solve everything, don't be afraid ;-). We do have an internal in source gettext. Couldn't we just support that and live all other version aside? I'd rather like to get rid of that instead of completely relying on it and the need to keep it up to date. Maybe this would simplify the code a bit, dunno. Yes, it would, but it would not make it faster unless we modify it to cache multiple languages. The latter would create even more maintenance problems than we already have (out internal gettext is almost always out of date, and I remember that some updates caused problems on some platforms). One other thing that may be changed also is the way we load po files. It seems that there is some caching done inside gettext but it is disabled as soon as we change the po file. This is explained here: http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152 Maybe there's something wrong in the way we load the po files? Just asking to you knowledgeable people... I have no idea. I think that basicxally Jean-Marc is right that gettext was just not designed for frequent multiple language switching like we do. Georg
Do we support user defined layout?
Hello,
I finally know what is causing the slowness in updateLabel() on Windows!
The culprit is the translation done in Buffer::translateLabel():
docstring const Buffer::translateLabel(docstring const & label) const
{
if (support::isAscii(label))
// Probably standard layout, try to translate
return B_(to_ascii(label));
else
// This must be a user defined layout. We cannot translate
// this, since gettext accepts only ascii keys.
return label;
}
If I short-cuts this and return label (thus avoiding the gettext
translation), the paragraph breaking is very fast!
My idea would be to cache the translated text instead of translating
each time updateLabels() is called. But I am not sure I can do that on
untranslatable layouts.
I seem to remember from previous discussions that user-defined layout
command are not allowed any more.
So question to Georg, Jose, JMarc or ???:
How is this layout business related to that translateLabel() method? Can
I just ditch the method and use _() directly?
Another idea would be to translate the layout on first loading and use
that later on. Then we could continue to support untranslatable layouts.
I am not sure I got everything right so please explain me.
Abdel.
Re: Do we support user defined layout?
Abdelrazak Younes wrote: > My idea would be to cache the translated text instead of translating > each time updateLabels() is called. But I am not sure I can do that on > untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. > I seem to remember from previous discussions that user-defined layout > command are not allowed any more. Wrong. That would mean that nobody could create custom layouts in ~/.lib/layouts. There is no reason to forbid that. > How is this layout business related to that translateLabel() method? Can > I just ditch the method and use _() directly? No. _() would assert if a user defined layout contained a label name with non-ascii characters. That was the very reason to introduce Buffer::translateLabel. Besides the slowness is in _(), not the additional ascii test. > Another idea would be to translate the layout on first loading and use > that later on. Then we could continue to support untranslatable layouts. That would be problematic since the translation depends on the buffer language. But if you introduce a cache at buffer level it should work. > I am not sure I got everything right so please explain me. I hope it is clear now. Georg
Re: Do we support user defined layout?
Georg Baum wrote: > Abdelrazak Younes wrote: >> I seem to remember from previous discussions that user-defined layout >> command are not allowed any more. > > Wrong. That would mean that nobody could create custom layouts in > ~/.lib/layouts. There is no reason to forbid that. Now I see that I overread 'command'. What you mean are style names, but what is translated here are static labels. Georg
Re: Do we support user defined layout?
Georg Baum wrote:
Georg Baum wrote:
Abdelrazak Younes wrote:
I seem to remember from previous discussions that user-defined layout
command are not allowed any more.
Wrong. That would mean that nobody could create custom layouts in
~/.lib/layouts. There is no reason to forbid that.
Now I see that I overread 'command'. What you mean are style names, but what
is translated here are static labels.
Yes, that's what I meant. The problem is that I don't know what's there
to translate. In a koma-script or article layout, here is what
layout->labelstring() gives inside 'expandLabel()':
raw fmt: \arabic{chapter}.\arabic{section}
raw fmt: @[EMAIL PROTECTED]
Am I right here that there is nothing to translate?
For the 'report' layout' we can have something like this:
raw fmt: Chapter \arabic{chapter}
raw fmt: \arabic{chapter}.\arabic{section}
Here, is it right that the only translatable part is the word "Chapter"?
If yes, wouldn't it be possible to translate that word once and for all
at layout loading? Then we could avoid translateLabel() altogether.
Abdel.
Re: Do we support user defined layout?
> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes: Abdelrazak> My idea would be to cache the translated text instead of Abdelrazak> translating each time updateLabels() is called. But I am Abdelrazak> not sure I can do that on untranslatable layouts. This is a problem with multiple languages. I _think_ the problem is that switching language is expensive with gettext (I am not sure gettext is designed to change languages very often, like we do). If this is the case, then we could cache string/lang pairs. JMarc
Re: Do we support user defined layout?
> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:
Abdelrazak> Yes, that's what I meant. The problem is that I don't know
Abdelrazak> what's there to translate. In a koma-script or article
Abdelrazak> layout, here is what
layout-> labelstring() gives inside 'expandLabel()':
Abdelrazak> raw fmt: \arabic{chapter}.\arabic{section} raw fmt:
Abdelrazak> @[EMAIL PROTECTED]
Abdelrazak> Am I right here that there is nothing to translate?
No. Look for example at this eu.po snippet:
msgid "Chapter \\arabic{chapter}"
msgstr "\\arabic{chapter}. kapitulua"
Basque does not have the same conventions as we do. These are rare
cases, but they exist.
JMarc
Re: Do we support user defined layout?
Jean-Marc Lasgouttes wrote:
"Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:
Abdelrazak> Yes, that's what I meant. The problem is that I don't know
Abdelrazak> what's there to translate. In a koma-script or article
Abdelrazak> layout, here is what
layout-> labelstring() gives inside 'expandLabel()':
Abdelrazak> raw fmt: \arabic{chapter}.\arabic{section} raw fmt:
Abdelrazak> @[EMAIL PROTECTED]
Abdelrazak> Am I right here that there is nothing to translate?
No. Look for example at this eu.po snippet:
msgid "Chapter \\arabic{chapter}"
msgstr "\\arabic{chapter}. kapitulua"
Basque does not have the same conventions as we do. These are rare
cases, but they exist.
Oh... shit.
Abdel.
Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote:
Abdelrazak Younes wrote:
Jean-Marc Lasgouttes wrote:
"Abdelrazak" == Abdelrazak Younes
<[EMAIL PROTECTED]> writes:
Abdelrazak> My idea would be to cache the translated text instead of
Abdelrazak> translating each time updateLabels() is called. But I am
Abdelrazak> not sure I can do that on untranslatable layouts.
This is a problem with multiple languages. I _think_ the problem is
that switching language is expensive with gettext (I am not sure
gettext is designed to change languages very often, like we do).
I think you're right.
If this is the case, then we could cache string/lang pairs.
You mean at gettext.[Ch] level?
Or rather at messages.[Ch] level. As this is set to one unique language
Apparently, this is not true and I don't understand why. The language is
set at construction of the Message class. A language change in the
document will reset the Message object to the new language. Up to that
point I understand.
But, updateLabel() is using Messages::Pimpl::get() and this can change
the language at each call! This is because of this code that checks ans
sets environment variables:
docstring const get(string const & m) const
{
if (m.empty())
return from_ascii(m);
// In this order, see support/filetools.C:
string lang = getEnv("LC_ALL");
if (lang.empty()) {
lang = getEnv("LC_MESSAGES");
if (lang.empty()) {
lang = getEnv("LANG");
if (lang.empty())
lang = "C";
}
}
#ifdef HAVE_LC_MESSAGES
char const * lc_msgs = setlocale(LC_MESSAGES, lang_.c_str());
#endif
This last line in particular looks suspicious. Why do we set environment
variable at each translation request?
This other chunk of code rings a bell in the context of the hebrew RTL
bug (related to LC_TYPE):
// CTYPE controls what getmessage thinks what encoding the po file uses
char const * lc_ctype = setlocale(LC_CTYPE, NULL);
string oldCTYPE = lc_ctype ? lc_ctype : "";
I don't know anything about gettext but this stuff looks awfully
complicated. Couldn't we simplify the thing a bit?
Abdel.
Re: Do we support user defined layout?
Georg Baum wrote: Abdelrazak Younes wrote: My idea would be to cache the translated text instead of translating each time updateLabels() is called. But I am not sure I can do that on untranslatable layouts. They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. Do you mean in the Document settings? If yes, then we just need to reset the cache when this is changed. Another idea would be to translate the layout on first loading and use that later on. Then we could continue to support untranslatable layouts. That would be problematic since the translation depends on the buffer language. But if you introduce a cache at buffer level it should work. Problem is that a single Buffer can presents multiple languages as explained in the other part of the thread. I am not sure I got everything right so please explain me. I hope it is clear now. Clearer yes, thanks. Abdel.
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote: > This last line in particular looks suspicious. Why do we set environment > variable at each translation request? Because the environment variable determines what language gettext will use, and we don't know what other message was translated before. To my knowledge no library call of gettext exists that takes the language as a function argument rather than an environment variable. > This other chunk of code rings a bell in the context of the hebrew RTL > bug (related to LC_TYPE): > > // CTYPE controls what getmessage thinks what encoding the po file uses > char const * lc_ctype = setlocale(LC_CTYPE, NULL); > string oldCTYPE = lc_ctype ? lc_ctype : ""; But setting LC_TYPE is needed for gettext. > I don't know anything about gettext but this stuff looks awfully > complicated. Couldn't we simplify the thing a bit? I don't think so. This awfully complicated stuff is the result of many bug fixes and a lot of work of several people. It is so complicated because some gettext versions are broken (especially on windows). I once tried to simplify this but only made Michael Gerz angry who suddenly discovered lots of bugs. Unless you really want to spend a lot of time yo learn all aspects about gettext I suggest that you leave this alone. Georg
Re: Do we support user defined layout?
Abdelrazak Younes wrote: > Georg Baum wrote: >> They are not different than the translatable ones. The only problem I can >> imagine is that the cache could get out of date if somebody changes the >> layout file. > > Do you mean in the Document settings? If yes, then we just need to reset > the cache when this is changed. No, I mean if somebody changes the layout file on disk. I don't know whether we read layout files only on startup, or always if a buffer is loaded, so I don't know whether this is already a problem or not. > Problem is that a single Buffer can presents multiple languages as > explained in the other part of the thread. Indeed. Then maybe a cache per layout but with different languages would work. Georg
Re: Translation madness! (was Re: Do we support user defined layout?)
Georg Baum wrote: Abdelrazak Younes wrote: This last line in particular looks suspicious. Why do we set environment variable at each translation request? Because the environment variable determines what language gettext will use, and we don't know what other message was translated before. To my knowledge no library call of gettext exists that takes the language as a function argument rather than an environment variable. [...] Unless you really want to spend a lot of time yo learn all aspects about gettext I suggest that you leave this alone. OK, thanks for the explanation. I am not pretending that I will solve everything, don't be afraid ;-). We do have an internal in source gettext. Couldn't we just support that and live all other version aside? Maybe this would simplify the code a bit, dunno. One other thing that may be changed also is the way we load po files. It seems that there is some caching done inside gettext but it is disabled as soon as we change the po file. This is explained here: http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152 Maybe there's something wrong in the way we load the po files? Just asking to you knowledgeable people... Abdel.
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote:
Georg Baum wrote:
Abdelrazak Younes wrote:
This last line in particular looks suspicious. Why do we set environment
variable at each translation request?
Because the environment variable determines what language gettext will
use,
and we don't know what other message was translated before. To my
knowledge
no library call of gettext exists that takes the language as a function
argument rather than an environment variable.
[...]
Unless you really want to spend a lot of time yo learn all aspects about
gettext I suggest that you leave this alone.
OK, thanks for the explanation. I am not pretending that I will solve
everything, don't be afraid ;-). We do have an internal in source
gettext. Couldn't we just support that and live all other version aside?
Maybe this would simplify the code a bit, dunno.
One other thing that may be changed also is the way we load po files. It
seems that there is some caching done inside gettext but it is disabled
as soon as we change the po file. This is explained here:
http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152
Maybe there's something wrong in the way we load the po files? Just
asking to you knowledgeable people...
At this link they say something interesting WRT changing language:
http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC155
I don't see any use of this '_nl_msg_cat_cntr' variable in the code.
Maybe that's a solution. Did you tried that in the past?
10.5 Being a gettext grok
To fully exploit the functionality of the GNU gettext library it is
surely helpful to read the source code. But for those who don't want to
spend that much time in reading the (sometimes complicated) code here is
a list comments:
* Changing the language at runtime
For interactive programs it might be useful to offer a selection
of the used language at runtime. To understand how to do this one need
to know how the used language is determined while executing the gettext
function. The method which is presented here only works correctly with
the GNU implementation of the gettext functions.
In the function dcgettext at every call the current setting of
the highest priority environment variable is determined and used.
Highest priority means here the following list with decreasing priority:
1. LANGUAGE
2. LC_ALL
3. LC_xxx, according to selected locale
4. LANG
Afterwards the path is constructed using the found value and the
translation file is loaded if available.
What is now when the value for, say, LANGUAGE changes. According
to the process explained above the new value of this variable is found
as soon as the dcgettext function is called. But this also means the
(perhaps) different message catalog file is loaded. In other words: the
used language is changed.
But there is one little hook. The code for gcc-2.7.0 and up provides
some optimization. This optimization normally prevents the calling of
the dcgettext function as long as no new catalog is loaded. But if
dcgettext is not called the program also cannot find the LANGUAGE
variable be changed (see section 10.2.7 Optimization of the *gettext
functions). A solution for this is very easy. Include the following code
in the language switching function.
/* Change language. */
setenv ("LANGUAGE", "fr", 1);
/* Make change known. */
{
extern int _nl_msg_cat_cntr;
++_nl_msg_cat_cntr;
}
Abdel.
Re: Do we support user defined layout?
On Fri, 12 Jan 2007, Georg Baum wrote: Abdelrazak Younes wrote: Georg Baum wrote: They are not different than the translatable ones. The only problem I can imagine is that the cache could get out of date if somebody changes the layout file. Do you mean in the Document settings? If yes, then we just need to reset the cache when this is changed. No, I mean if somebody changes the layout file on disk. I don't know whether we read layout files only on startup, or always if a buffer is loaded, so I don't know whether this is already a problem or not. When I messed with layouts as a user, LyX only read layout files on startup. So that's how the older versions did it and AFAIK this has not changed. However, I really hope LyX will be able to to re-read the layout files in the future so I think it'd be bad to add a constraint. cheers /Christian -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr
Re: Translation madness! (was Re: Do we support user defined layout?)
Abdelrazak Younes wrote: > OK, thanks for the explanation. I am not pretending that I will solve > everything, don't be afraid ;-). We do have an internal in source > gettext. Couldn't we just support that and live all other version aside? I'd rather like to get rid of that instead of completely relying on it and the need to keep it up to date. > Maybe this would simplify the code a bit, dunno. Yes, it would, but it would not make it faster unless we modify it to cache multiple languages. The latter would create even more maintenance problems than we already have (out internal gettext is almost always out of date, and I remember that some updates caused problems on some platforms). > One other thing that may be changed also is the way we load po files. It > seems that there is some caching done inside gettext but it is disabled > as soon as we change the po file. This is explained here: > > http://www.gnu.org/software/gettext/manual/html_mono/gettext.html#SEC152 > > Maybe there's something wrong in the way we load the po files? Just > asking to you knowledgeable people... I have no idea. I think that basicxally Jean-Marc is right that gettext was just not designed for frequent multiple language switching like we do. Georg
