Spaces in Bidi part #2

2007-06-15 Thread Dov Feldstern
Okay, I'm starting a new thread for this issue. This is now bug 3870 in 
bugzilla (http://bugzilla.lyx.org/show_bug.cgi?id=3870).


The problem: typing spaces at RTL/LTR boundaries does not always do what 
the user expects/wants.


(The reason I opened a new bug, is that this really a different issue 
than what we have solved up until now (bug 3789): that was mainly about 
getting the GUI and backend to match. Now that they match, we can deal 
with the real issue. The only thing we have done for the real issue 
so far, is that in normal typing --- i.e., typing in the logical order 
--- the language of the space will be set correctly, regardless of 
whether or not the user switches languages before or after typing the 
space. We may decide, in the end, that that's not necessary, either...)


Note that there are two separate issues:

1) What the end result should look like (at the buffer / latex output level)
2) How we achieve that result, at the same time making the 
user-experience as simple as possible


Note that (1) is totally unrelated to visual/logical movement --- 
there's no cursor at all in the end result. I'm quite sure that also (2) 
is orthogonal to the question of visual movement, but I'm not 100% sure...


Obviously, (2) can only be solved after (1), because we first need to 
know what we want to achieve. So I'm focusing on (1) for now:


There there are some subtleties to this issue which I'm not sure that 
everyone appreciates, and which make a solution to this not as trivial 
as some of you seem to think it should be.


The hard part is to make sure that we maintain logical correctness of 
the resulting text. Logical correctness is important, because if the 
text is *not* logically correct, then you get hidden problems: usually 
the text looks fine, so the user thinks there's no problem. Bit in 
certain situations, problems suddenly appear, but by then the user isn't 
aware of them and doesn't notice.


For example:

I've been saying all along that the *correct* way is that spaces at 
RTL/LTR boundaries be in the language (direction, really) of the 
paragraph. Any other situation is logically incorrect. Attached is a lyx 
document demonstrating this. (Now is the time to look at the file, there 
are explanations about it inside it itself. For those of you who don't 
have hebrew in latex, I'm attaching the dvi, too).


I think this demonstrates the necessity of maintaining logical 
correctness. So now we have to see how we achieve (2). However, we have 
the following rule: any magic has to happen in the GUI; i.e., the LyX 
buffer should already represent a logically correct situation; i.e., 
when we generate the latex, we should not have to apply any bidi logic 
at all --- just make what has *already* been marked as RTL, as RTL, and 
what's LTR as LTR.


So now we can discuss how we go about (2).

Hope this clarifies the situation...
Dov


spaces_at_bidi_boundary.dvi
Description: TeX dvi file


spaces_at_bidi_boundary.lyx
Description: application/lyx


Spaces in Bidi part #2

2007-06-15 Thread Dov Feldstern
Okay, I'm starting a new thread for this issue. This is now bug 3870 in 
bugzilla (http://bugzilla.lyx.org/show_bug.cgi?id=3870).


The problem: typing spaces at RTL/LTR boundaries does not always do what 
the user expects/wants.


(The reason I opened a new bug, is that this really a different issue 
than what we have solved up until now (bug 3789): that was mainly about 
getting the GUI and backend to match. Now that they match, we can deal 
with the "real" issue. The only thing we have done for the "real issue" 
so far, is that in "normal typing" --- i.e., typing in the logical order 
--- the language of the space will be set correctly, regardless of 
whether or not the user switches languages before or after typing the 
space. We may decide, in the end, that that's not necessary, either...)


Note that there are two separate issues:

1) What the end result should look like (at the buffer / latex output level)
2) How we achieve that result, at the same time making the 
user-experience as simple as possible


Note that (1) is totally unrelated to visual/logical movement --- 
there's no cursor at all in the end result. I'm quite sure that also (2) 
is orthogonal to the question of visual movement, but I'm not 100% sure...


Obviously, (2) can only be solved after (1), because we first need to 
know what we want to achieve. So I'm focusing on (1) for now:


There there are some subtleties to this issue which I'm not sure that 
everyone appreciates, and which make a solution to this not as trivial 
as some of you seem to think it should be.


The hard part is to make sure that we maintain logical correctness of 
the resulting text. Logical correctness is important, because if the 
text is *not* logically correct, then you get "hidden" problems: usually 
the text looks fine, so the user thinks there's no problem. Bit in 
certain situations, problems suddenly appear, but by then the user isn't 
aware of them and doesn't notice.


For example:

I've been saying all along that the *correct* way is that spaces at 
RTL/LTR boundaries be in the language (direction, really) of the 
paragraph. Any other situation is logically incorrect. Attached is a lyx 
document demonstrating this. (Now is the time to look at the file, there 
are explanations about it inside it itself. For those of you who don't 
have hebrew in latex, I'm attaching the dvi, too).


I think this demonstrates the necessity of maintaining logical 
correctness. So now we have to see how we achieve (2). However, we have 
the following rule: any "magic" has to happen in the GUI; i.e., the LyX 
buffer should already represent a logically correct situation; i.e., 
when we generate the latex, we should not have to apply any bidi logic 
at all --- just make what has *already* been marked as RTL, as RTL, and 
what's LTR as LTR.


So now we can discuss how we go about (2).

Hope this clarifies the situation...
Dov


spaces_at_bidi_boundary.dvi
Description: TeX dvi file


spaces_at_bidi_boundary.lyx
Description: application/lyx