Re: [XeTeX] XeTeX and "ignore sub" substitution rules
> > On further testing, I don't think Apostolos's font works with XeTeX > either. It may appear to at first glance, just because the effect of the > "ignore sub" rules in that font is very subtle, but if I modify the > alternate glyphs to be more obviously different, it's clear that they are > being put in in cases where the rules say they shouldn't. > The font is OK. I have done some testing and it does not produce correect results with the latest version of XeTeX while it did produce the expected results with say XeTeX that comes with TeXLive 2009. I will give more details later on. A.S. -- Apostolos Syropoulos Xanthi, Greece -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, 30 Nov 2011, msk...@ansuz.sooke.bc.ca wrote: > In the Korean fonts I'm currently working on, some syllables are converted > to single precomposed glyphs by ligature substitution, and others are > built up by overlaying zero-width glyphs, and the difference between the I was interested to test just what the effects of trying to colour part of a Korean syllable with my fonts actually were, so I did some experiments. Demonstration attached. -- Matthew Skala msk...@ansuz.sooke.bc.ca People before principles. http://ansuz.sooke.bc.ca/ colordemo.pdf Description: Adobe PDF document -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, 30 Nov 2011, Khaled Hosny wrote:
> processed by the layout engine, which would require keeping account of
> character to glyph mapping (which is doable, all text editing GUI's have
> to do it).
It will, of course, have to do something sensible (even if that just means
complain) should the desired location of a reinserted special end up in
the middle of a ligature.
Suppose someone types
f\textcolor{red}{f}
and the font tries to replace the two fs with an ff ligature. In that
case there is no sensible way to colour just the second one red.
Changing the colour of a mark and not the base is different from changing
the colour of just part of a ligature, and the system can certainly
distinguish mark-to-base from ligatures; but especially if colour is
handled by a special that the TeX engine only understands as an opaque
blob of data, it'll be hard for the engine to know what is the right thing
to do with the above.
In the Korean fonts I'm currently working on, some syllables are converted
to single precomposed glyphs by ligature substitution, and others are
built up by overlaying zero-width glyphs, and the difference between the
two ways of drawing syllables should ideally not be visible to the user.
(This approach is driven by Unicode, which specifies roughly 11000 code
points for entire Korean syllables, but a more complicated way of
expressing the several hundred thousand imaginable syllables that do not
have code points of their own.) The zero-width glyphs also change their
shapes contextually. Suppose someone tries to change the colour of just
one of the sub-glyphs in what would otherwise be a precomposed syllable.
Well, if that breaks the glyph sequence for the purposes of the ligature
substitution, it's actually a good thing. It means my font will fall
back on the overlaid zero-width glyphs, and then just the one the user
picked will change colour, and the user will get what they wanted. The
layout of the overlaid combination won't be quite as good as if the font
had been allowed to use a precomposed glyph, but there isn't really any
better way it could work.
Trouble is, the overlaid zero-width glyph method also depends on
contextual substitutions (because there are actually six different layouts
for an overlaid syllable, used to approximate the more nuanced layout
decisions that go into the precomposed glyphs). If the colour special
interrupts the glyph sequence for the purpose of those contextual
substitutions, then the font will end up falling all the way back to its
default layout (or, even worse, different layout guesses for different
parts of the same syllable) and the result will be unacceptable. The
ideal for my fonts would be if colour specials interrupted the glyph
sequence in the liga feature - thus preventing ligature substitution from
occurring - but not in the ccmp, ljmo, or vjmo features. What would be
ideal in some other font might be different from that.
I don't know the right answer to how it should work, but I'm highlighting
this issue just because I hope anyone who tries to implement a solution
will think carefully about the consequences for multiple cases.
--
Matthew Skala
msk...@ansuz.sooke.bc.ca People before principles.
http://ansuz.sooke.bc.ca/
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 09:47:31PM +0100, Keith J. Schultz wrote: > Hi Khaled, > > I am afraid you are completely misconceived about what the subject matter is > here. > 1) adding a diacritic mark(glyph) is composing a a glyph, You are able > to output it > on its own. > > 2) There is a difference between the glyph ä and adding the diactirc > mark of umlaut to > an a. > a) in the first case you can use two different colors because it > is one glyph. > b) in the second you can use two different colors because you > two glyph in order > two compose the ä glyph. > > So we are talking about composed glyphs whether you realize it or not. No we are not: 1) adding a mark to a glyphs is one thing, and composing them into a single glyph is another thing (in Latin fonts the later is a common result of the former, but that is mere implementation detail). The fully vowelled Arabic word مُحَمَّدٌ have a mark (or even two) in each character but no font will compose each into a single glyph (not a font build by a sane person). The same is even true for versatile Latin fonts that allow proper positioning of arbitrary combinations (e.g. many fonts from SIL). 2) the OP is developing his own font, which is neither covering Latin glyphs nor using precomposed glyphs. Regards, Khaled > > regards > Keith. > > > Am 30.11.2011 um 13:56 schrieb Khaled Hosny: > > > On Wed, Nov 30, 2011 at 11:10:11AM +0100, Keith J. Schultz wrote: > >> Hi All, > >> > >> I jump back in. I will cite anybody because what has been said is correct. > >> > >> But, > >> > >>1) trying to compare a browser, XeTex engine and LuaTeX will not help > >>as they have different methods of composing their output. > >>That is how they compose and position their glyphs. > > > > As far as OpenType processing is concerned they all should give the same > > result, else there is a bug somewhere. > > > >>2) Most important a composed Unicode glyph is supposed to be just one > >> color!! > > > > No one talked about composed glyphs (certainly not the OP), it was just > > a marginal and unrelated issue to the problem being discussed; coloring > > combining marks without breaking OpenType mark positioning. > > > >>3) Once you start using color a Unicode composed glyph you no longer > >> are positioning > >> a single composed glyph, but two or more glyphs. > > > > So? Again, coloring components of composed glyphs is not what is being > > discussed here. > > > >>3.a) color designed in TeX at.al is designed be applied to a box and > >> not glyphs!! > > > > I've hard time understanding what this mean or how the difference, if > > any, is material. AFAIK, TeX knows nothing about color, it handled just > > like any other driver special which TeX makes no effort to interpret not > > to mention "applying" it. > > > >> The question remains how to position the composed glyphs and where and how > >> the color attribute > >> is added to the output. > >> > >> There are therefore two solutions: > >>1) The Tex way: > >>create a macro to compose the glyph and do the positioning and > >> coloring. > >> > >>2) The developer way: > >>change the engine so that it firsts generates the composed > >> glyph and then goes back > >>and then applies color to the different glyphs. > > > > 3) The post-2000s way; use/build an OpenType font with proper combining > > mark positioning and apply the colors to individual glyphs à la what > > FireFox/LuaTeX and may be many other does. > > > > Regards, > > Khaled > > > > > > -- > > Subscriptions, Archive, and List information, etc.: > > http://tug.org/mailman/listinfo/xetex > > > > > -- > Subscriptions, Archive, and List information, etc.: > http://tug.org/mailman/listinfo/xetex -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 02:17:07PM -0500, Aleksandr Andreev wrote:
> Khaled Hosny writes:
>
> > use/build an OpenType font with proper combining
> > mark positioning and apply the colors to individual glyphs à la what
> > FireFox/LuaTeX and may be many other does.
>
> What I get out of this comment and the post by Heiko Oberdiek is that
> this should be possible to accomplish in XeTeX.
>
> My next question is this: is this then an issue with XeTeX, dvipdfmx,
> ICU or Ghostscript?
>
> I am just trying to understand here the workflow whereby a TeX source
> document becomes PDF in XeTeX and which component breaks the proper
> positioning.
My understanding is that XeTeX reads input, converts characters into a
special "native word" node which is then processed by the layout engine
and converted into and array of positioned glyphs. Color is inserted
using \special{} command, when XeTeX encounters any non character
command is stops collecting characters at that positions, so the input
A\special{pdf:color stuff}0308 is split into two "native word" nodes
by the \special{} command, each processed by the layout engine
separately i.e. it sees an A on its own and an 0308 on its own,
which means no OpenType interaction would take place (mark positioning,
ligaturing, whatever).
A solution might involve e.g. not splitting characters at specials
instead, the special and its position in the text stream would be saved
somehow, and then inserted back in the glyph array somehow after being
processed by the layout engine, which would require keeping account of
character to glyph mapping (which is doable, all text editing GUI's have
to do it).
How easy/hard is that is beyond me.
Regards,
Khaled
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Khaled, I am afraid you are completely misconceived about what the subject matter is here. 1) adding a diacritic mark(glyph) is composing a a glyph, You are able to output it on its own. 2) There is a difference between the glyph ä and adding the diactirc mark of umlaut to an a. a) in the first case you can use two different colors because it is one glyph. b) in the second you can use two different colors because you two glyph in order two compose the ä glyph. So we are talking about composed glyphs whether you realize it or not. regards Keith. Am 30.11.2011 um 13:56 schrieb Khaled Hosny: > On Wed, Nov 30, 2011 at 11:10:11AM +0100, Keith J. Schultz wrote: >> Hi All, >> >> I jump back in. I will cite anybody because what has been said is correct. >> >> But, >> >> 1) trying to compare a browser, XeTex engine and LuaTeX will not help >> as they have different methods of composing their output. >> That is how they compose and position their glyphs. > > As far as OpenType processing is concerned they all should give the same > result, else there is a bug somewhere. > >> 2) Most important a composed Unicode glyph is supposed to be just one >> color!! > > No one talked about composed glyphs (certainly not the OP), it was just > a marginal and unrelated issue to the problem being discussed; coloring > combining marks without breaking OpenType mark positioning. > >> 3) Once you start using color a Unicode composed glyph you no longer >> are positioning >> a single composed glyph, but two or more glyphs. > > So? Again, coloring components of composed glyphs is not what is being > discussed here. > >> 3.a) color designed in TeX at.al is designed be applied to a box and >> not glyphs!! > > I've hard time understanding what this mean or how the difference, if > any, is material. AFAIK, TeX knows nothing about color, it handled just > like any other driver special which TeX makes no effort to interpret not > to mention "applying" it. > >> The question remains how to position the composed glyphs and where and how >> the color attribute >> is added to the output. >> >> There are therefore two solutions: >> 1) The Tex way: >> create a macro to compose the glyph and do the positioning and >> coloring. >> >> 2) The developer way: >> change the engine so that it firsts generates the composed >> glyph and then goes back >> and then applies color to the different glyphs. > > 3) The post-2000s way; use/build an OpenType font with proper combining > mark positioning and apply the colors to individual glyphs à la what > FireFox/LuaTeX and may be many other does. > > Regards, > Khaled > > > -- > Subscriptions, Archive, and List information, etc.: > http://tug.org/mailman/listinfo/xetex -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Heiko, and others
On 30/11/2011, at 8:56 PM, Heiko Oberdiek wrote:
> The PDF stuff:
>
> % without color:
> 0 -99.63 Td[<0024>]TJ
> 54.19 15.57 Td[<0301>]TJ
>
> % with color:
> -54.19 -115.2 Td[<0024>]TJ
> ET 1 0 0 RG 1 0 0 rg BT /F1 99.626 Tf
> 59.3 -278.84 Td[<0301>]TJ
>
> % with color via \special{pdf:code ...}:
> 0 -378.46 Td[<0024>]TJ
> ET 1 0 0 rg 1 0 0 RG BT /F1 99.626 Tf
> 59.3 -378.46 Td[<0301>]TJ
>
> It seems that in XeTeX the color cannot be inserted without
> breaking the PDF text sections (BT...ET). \special{pdf:literal direct ...}
> (or short \special{pdf:code ...}) is not equivalent to
> \pdfliteral direct{...} but rather to \pdfliteral page{...}
> that ends a text section in the PDF output.
>
> If someone want's this issue fixed, make a feature request
> for XeTeX/xdvipdfmx to provide a real "\pdfliteral direct"/
> "direct mode for colors" without breaking text sections.
In the experimental version of pdfTeX with support for Tagged PDF,
we need a similar kind of variant to \pdfliteral .
It is called 'enclose'.
Here is how, in the coding of the corresponding pdftex.web
the various options are read, following an occurrence
of \pdfliteral :
@ @=
begin
check_pdfoutput("\pdfliteral", true);
new_whatsit(pdf_literal_node, write_node_size);
if scan_keyword("direct") then
pdf_literal_mode(tail) := direct_always
else if scan_keyword("page") then
pdf_literal_mode(tail) := direct_page
else if scan_keyword("enclose") then
pdf_literal_mode(tail) := enclose
else
pdf_literal_mode(tail) := set_origin;
scan_pdf_ext_toks;
pdf_literal_data(tail) := def_ref;
end
and here is the documentation on those options:
@# {data structure for \.{\\pdfliteral}; node size = 2}
@d pdf_literal_data(#) == link(#+1) {data}
@d pdf_literal_mode(#) == info(#+1) {mode of resetting the text matrix
while writing data to the page stream}
@# {modes of setting the current transformation matrix (CTM)}
@d set_origin == 0 {end text (ET) if needed, set CTM to current
point}
@d direct_page == 1 {end text (ET) if needed, but don't change the
CTM}
@d direct_always == 2 {don't end text, don't change the CTM}
@d scan_special== 3 {look into special text}
@d enclose == 4 {like |direct_always|, but end current string
and sync pos}
The 'enclose' option is used to position the 'BDC' and 'EMC'
as closely around the text snippets as can be achieved,
without breaking the BT ... ET.
This seems to be the same kind of requirement here.
It would be nice if any extension to literal specials in xdvipdfmx
used the same keywords for similar functionality.
>
> Yours sincerely
> Heiko Oberdiek
Hope this helps,
Ross
Ross Moore ross.mo...@mq.edu.au
Mathematics Department office: E7A-419
Macquarie University tel: +61 (0)2 9850 8955
Sydney, Australia 2109 fax: +61 (0)2 9850 8114
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Khaled Hosny writes: > use/build an OpenType font with proper combining > mark positioning and apply the colors to individual glyphs à la what > FireFox/LuaTeX and may be many other does. What I get out of this comment and the post by Heiko Oberdiek is that this should be possible to accomplish in XeTeX. My next question is this: is this then an issue with XeTeX, dvipdfmx, ICU or Ghostscript? I am just trying to understand here the workflow whereby a TeX source document becomes PDF in XeTeX and which component breaks the proper positioning. Thanks, Aleks -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 11:10:11AM +0100, Keith J. Schultz wrote: > Hi All, > > I jump back in. I will cite anybody because what has been said is correct. > > But, > > 1) trying to compare a browser, XeTex engine and LuaTeX will not help > as they have different methods of composing their output. > That is how they compose and position their glyphs. As far as OpenType processing is concerned they all should give the same result, else there is a bug somewhere. > 2) Most important a composed Unicode glyph is supposed to be just one > color!! No one talked about composed glyphs (certainly not the OP), it was just a marginal and unrelated issue to the problem being discussed; coloring combining marks without breaking OpenType mark positioning. > 3) Once you start using color a Unicode composed glyph you no longer > are positioning >a single composed glyph, but two or more glyphs. So? Again, coloring components of composed glyphs is not what is being discussed here. > 3.a) color designed in TeX at.al is designed be applied to a box and > not glyphs!! I've hard time understanding what this mean or how the difference, if any, is material. AFAIK, TeX knows nothing about color, it handled just like any other driver special which TeX makes no effort to interpret not to mention "applying" it. > The question remains how to position the composed glyphs and where and how > the color attribute > is added to the output. > > There are therefore two solutions: > 1) The Tex way: > create a macro to compose the glyph and do the positioning and > coloring. > > 2) The developer way: > change the engine so that it firsts generates the composed > glyph and then goes back > and then applies color to the different glyphs. 3) The post-2000s way; use/build an OpenType font with proper combining mark positioning and apply the colors to individual glyphs à la what FireFox/LuaTeX and may be many other does. Regards, Khaled -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi All, I jump back in. I will cite anybody because what has been said is correct. But, 1) trying to compare a browser, XeTex engine and LuaTeX will not help as they have different methods of composing their output. That is how they compose and position their glyphs. 2) Most important a composed Unicode glyph is supposed to be just one color!! 3) Once you start using color a Unicode composed glyph you no longer are positioning a single composed glyph, but two or more glyphs. 3.a) color designed in TeX at.al is designed be applied to a box and not glyphs!! The question remains how to position the composed glyphs and where and how the color attribute is added to the output. There are therefore two solutions: 1) The Tex way: create a macro to compose the glyph and do the positioning and coloring. 2) The developer way: change the engine so that it firsts generates the composed glyph and then goes back and then applies color to the different glyphs. Such a change most likely will not happen for XeTeX, as the development has staled or if you wish is frozen. LuaTex is still under development and can be adjusted. regards Keith. -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 05:10:07AM +0200, Khaled Hosny wrote:
> > A\textcolor{red}{0308}\\ % (3)
> It seems Charis SIL composes accented glyphs, try Gentium Basic instead
> (GenBasR.ttf).
Thanks, the result is correct, the box contents after luacolor's
processing is:
\EU2/GentiumBasic(0)/m/n/100 A
\pdfliteral direct"1 0 0 rg 1 0 0 RG"
\EU2/GentiumBasic(0)/m/n/100 Ì
The part in the PDF with/without color:
% without color:
1 0 0 1 72 590.679 Tm [<0024>]TJ
1 0 0 1 126.191 606.246 Tm [<0301>]TJ
% with color:
1 0 0 1 72 491.053 Tm [<0024>]TJ
1 0 0 rg 1 0 0 RG
1 0 0 1 126.191 506.619 Tm [<0301>]TJ
In case of XeTeX:
.\EU1/GentiumBasic(0)/m/n/100 A
.\special{color push rgb 1 0 0}
.\EU1/GentiumBasic(0)/m/n/100 Ì210
.\special{color pop}
The PDF stuff:
% without color:
0 -99.63 Td[<0024>]TJ
54.19 15.57 Td[<0301>]TJ
% with color:
-54.19 -115.2 Td[<0024>]TJ
ET 1 0 0 RG 1 0 0 rg BT /F1 99.626 Tf
59.3 -278.84 Td[<0301>]TJ
% with color via \special{pdf:code ...}:
0 -378.46 Td[<0024>]TJ
ET 1 0 0 rg 1 0 0 RG BT /F1 99.626 Tf
59.3 -378.46 Td[<0301>]TJ
It seems that in XeTeX the color cannot be inserted without
breaking the PDF text sections (BT...ET). \special{pdf:literal direct ...}
(or short \special{pdf:code ...}) is not equivalent to
\pdfliteral direct{...} but rather to \pdfliteral page{...}
that ends a text section in the PDF output.
If someone want's this issue fixed, make a feature request
for XeTeX/xdvipdfmx to provide a real "\pdfliteral direct"/
"direct mode for colors" without breaking text sections.
Yours sincerely
Heiko Oberdiek
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
