Re: [XeTeX] XeTeX bugs in bidirectional typesetting

[Javier Bezos]

Again, if you have an explicit list of issues with tex--xet that are
not shared with tex-xet it would be good if you could post them here.


And issues with the omega/luatex model, too. I resumed my work on
the basic bidi support for babel and they would be very useful.

Javier


--
Subscriptions, Archive, and List information, etc.:
 http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] XeTeX bugs in bidirectional typesetting

[Simon Cozens]

On 20/11/2016 12:35, Vafa Khalighi wrote:
> For the past 10 years I have reported numerous issues to the LuaTeX 
> and etex teams without any results but it is more than enough. I am 
> not going to waste time and energy doing useless things.

Well, it's not necessarily useless. If both engines are using the same
model, then you have twice as many communities available to fix bugs.
And it means the bug fixes and bidi expertise can be shared between the
two communities. I would also recommend XeTeX moving to the LuaTeX model
- and then fixing it!

> I put my time and energy into developing an engine that really has a
>  working bidi model. an engine which is developed by a native speaker
>  and meets the needs of people with real documents.

XeTeX has a lot of advantages in terms of opentype support, large set of
packages and mature community. It's a shame the bidi support is not
great; that is a known problem and there are not many people with the
expertise to make it work and do it well. As I understand it, the
problems with bidi were one of the reasons that Khaled stopped working
on XeTeX, which in a sense is a shame - he's exactly the sort of person
you need to get this right...

If you want an engine with a working bidi model, then you might want to
have a look at SILE. (https://github.com/simoncozens/sile) It uses the
Unicode bidi algorithm and so you get multilevel reordering without any
markup required. (See
https://github.com/simoncozens/sile/blob/master/examples/arabic.pdf) But
of course then you don't get the large set of packages and the mature
community...

S


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

[XeTeX] XeTeX bugs in bidirectional typesetting

[Vafa Khalighi]

Hi David


Sadly, I can not agree with you; it seems that I have completely different 
perspective on bidi model which is reasonable considering my experience and 
expertise on the matter.


LuaTeX (Omega) model is not a good development for XeTeX since you will find 
yourself surrounded by so many bugs and you waste a lot of time fixing them but 
finally you will have to give up simply because its fundamentally broken. The 
TeX-XeT model is the most stable bidi model so far with few limitations and 
bugs and in its current state it is far better than LuaTeX (Omega) model. I put 
my trust and faith in TeX-XeT for two main reasons: (1) It was written by 
Donald Knuth (2) The ParsiTeX system which is based on Knuth TeX-XeT system has 
been used over 20 years and it is the most bug-free bidi model.


It is therefore clear that of the currently implemented
possibilities the TeX-XeT model is more functional, having more
complete support for right to left text and more bug-free.

For the past 10 years I have reported numerous issues to the LuaTeX and etex 
teams without any results but it is more than enough. I am not going to waste 
time and energy doing useless things. I put my time and energy into developing 
an engine that really has a working bidi model. an engine which is developed by 
a native speaker and meets the needs of people with real documents.






--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] XeTeX bugs in bidirectional typesetting

[David Carlisle]

Vafa Khalighi wrote


>  A) Drop the TeX--XeT model entirely and switch to LuaTeX (Omega)
> bidi model
>
>
> Unfortunately Omega's bidi model has its own set of bugs so even if
> XeTeX goes this route, we need to fix quite some issues in its bidi
> model.

Long term I think that this would be a good development for xetex.
Having tex engines with two completely different models for directionality
causes real issues for formats and packages that need to build on the
base provided by the engines. It is clear that of the currently implemented
possibilities the omega/luatex model is more functional, having more
complete
support for right to left text, but also supporting vertical text as well.

If you have a list of issues it would be good to raise them (on the luatex
list
initially would presumably be best).


>
>
> B) Switch to the original TeX-XeT model by Knuth
>
>
> I would prefer this approach (considering my over 10 years of
> experience with both TeX--XeT and Omega bidi model) as it is more
> stable than Omega's bidi model. If I recall correctly, Khaled Hosny
> did this in 2013 and when I was testing most of the problems were
> fixed. The only issue was that the original TeX-XeT model adds \beginL
> ... \endL and that would cause some problem in math typesetting
> (sorry, I can not remember what exactly it was about but I remember it
> was reported by David Carlisle). For this problem, one can look at
> TeX--XeT and see how display math are done there and we could apply
> the same approach to TeX-XeT model.
>

I really could not recommend this, the extra nodes added by tex-xet
interfere with typesetting in ways that are very hard to work around.
Particularly but not limited to math mode.
The only way shown so far to avoid those issues is not to add the nodes,
which essentially leads to tex--xet  rather than tex-xet.


Again, if you have an explicit list of issues with tex--xet that are
not shared with tex-xet it would be good if you could post them here.


David


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

[XeTeX] XeTeX bugs in bidirectional typesetting

[Vafa Khalighi]

Hi


XeTeX uses TeX--XeT algorithm and has some serious problems in typesetting 
right to left text. About few years ago, I wrote to Peter Bre­it­en­lohner and 
we discussed together what needs to be fixed. Peter promised that he would fix 
the issues but unfortunately he passed away. I believe he at least fixed one of 
the bugs I reported to him.


Bidirectional typesetting is a very fundamental area which unfortunately has 
been ignored so far. I can think of two ways to improve the situation:


A) Drop the TeX--XeT model entirely and switch to LuaTeX (Omega) bidi model


Unfortunately Omega's bidi model has its own set of bugs so even if XeTeX goes 
this route, we need to fix quite some issues in its bidi model.


B) Switch to the original TeX-XeT model by Knuth


I would prefer this approach (considering my over 10 years of experience with 
both TeX--XeT and Omega bidi model) as it is more stable than Omega's bidi 
model. If I recall correctly, Khaled Hosny did this in 2013 and when I was 
testing most of the problems were fixed. The only issue was that the original 
TeX-XeT model adds \beginL ... \endL and that would cause some problem in math 
typesetting (sorry, I can not remember what exactly it was about but I remember 
it was reported by David Carlisle). For this problem, one can look at TeX--XeT 
and see how display math are done there and we could apply the same approach to 
TeX-XeT model.


Once this is done, we can also look at ParsiTeX source[1] which is also based 
on TeX--XeT and add few primitives which are essential for bidi such as 
\Rhalign (making RTL tables).




Conclusion: I am not going to sit and watch like before! if XeTeX bidi model is 
not improved until TeXLive 2017, I am going to fork XeTeX, rename it to 
something else and make the changes I need. This is indeed last time I try to 
talk about it with the XeTeX team. I have done it numerous times and each time 
the bidi problems were ignored.







[1] http://ctan.org/pkg/parsitex

CTAN: Package parsitex
ctan.org
Sugges­tions. Maybe you are in­ter­ested in the fol­low­ing pack­ages as well. 
nts: New Type­set­ting Sys­tem; vtex-free: T e X sys­tem and PDF sup­port ...







--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] bidi-problem --> problems with lines not starting in position 1 (even comments)

[Zdenek Wagner]

AFAIK polyglossia knows whether bidi is needed and loads it automatically.
The advantage is that it can change the font and language specific things.

But back to the problem. Missing \begin{document} means that something
tried to typeset a text. It is caused by an expansion of \everypar which is
redefined to issue this error message. Moving \begin{document} to an
earlier place removes the error message but does not solve the real error.
The error will only abeear later and will be stranger and less
understandable. You should find which definition tries to typeset a text
and why.

Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz

2016-11-19 6:19 GMT+01:00 Karljurgen Feuerherm :

> Hello Herb,
>
> That’s not quite the same thing, is it, Herb?
>
> >From the documentation of Polyglossia, p. 3*: "The current version of
> polyglossia makes use of some convenient macros defined in the etoolbox
> package by Philipp Lehmann. Being designed
> for XƎLATEX and LuaLATEX, it obviously also relies on fontspec by Will
> Robertson. For languages written from right to left, it needs the package
> bidi by Vafa Khaligi”.
>
> K
>
> * 2015/03/25 v1.42.4
> (pdf file generated on 25 March 2016)
>
> Karljürgen G. Feuerherm, PhD | 2-139 Department of History | Wilfrid
> Laurier University | 75 University Avenue West | Waterloo, Ontario N2L 3C5
>
>
>
>
>
> On 2016-11-18, 17:22, "XeTeX on behalf of Herbert Schulz"
>  wrote:
>
> >
> >Howdy,
> >
> >I don't know if it applies here but I thought most folks use polyglossia
> >rather than bidi with xelatex.
> >
> >Good Luck,
> >
> >Herb Schulz
> >(herbs at wideopenwest dot com)
> >
> >
> >
> >
> >--
> >Subscriptions, Archive, and List information, etc.:
> >  http://tug.org/mailman/listinfo/xetex
>
>
>
>
> --
> Subscriptions, Archive, and List information, etc.:
>   http://tug.org/mailman/listinfo/xetex
>


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex