Re: [XeTeX] Random number primitives

[Bruno Le Floch]

On 11/14/2016 04:47 AM, David Carlisle wrote:
> On 14 November 2016 at 09:22, Apostolos Syropoulos
>  wrote:
>>>
>>> Changing the source isn't the issue, are you offering also to help
>>> with user support to manage the change
>>> and explain to people why it's not possible to re-create documents
>>> that they created before they updated
>>> their tex system?
>>>
>>
>> By following this conservative line of thought, we should
>> not use OpenType fonts and stick to plain old but good Type 1
>> fonts!
> 
> No, not at all. There is a big difference between offering new functionality
> and offering existing functionality. when implementing existing tex features.
> 
> xetex (far more than luatex) follows tex very closely, so that in the
> main, if you are
> not using the extended features it works exactly like tex. All I am
> suggesting is that
> design philosophy should apply here.
> 
>> In fact, I would recommend dropping support for Type 1
>> fonts from XeTeX. If you want Type 1 fonts, use the ancient
>> engines.
> 
>>  Also, each TeX destribution comes with all these
>> TeX-based engines. So these people can always use their
>> ancient files with no problem!
> 
> Er no, That was entirely the point of my message.
> I was replying to your offer to change the sources of pdftex. If that were 
> done
> people would _not_ be able to use their ancient (or even not ancient)
> files and get the same results.
> 
> David
> 

I had time to look more carefully at the source code in pdftex.web.  It
works by producing a stream of 28-bit integers (details don't matter
here) and rescaling that to the user's requested interval.  This means
for instance that \pdfuniformdeviate 536870912 only produces even
numbers.  This seemingly innocuous loop loops forever.

\loop\ifodd\pdfuniformdeviate 536870912\relax\else\repeat

I don't think XeTeX should adopt this bug for the sake of compatibility.
 I will not have time to implement anything in Pascal nor C in the near
future.

Best regards,

Bruno


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Apostolos Syropoulos]

>
> No, not at all. There is a big difference between offering new >functionality
> and offering existing functionality. when implementing existing tex >features.
> 

Existing functionality is something that is already there
but this discussion is on because this functionality is
not included in XeTeX. Also, this is not a TeX feature but
a pdfTeX feature!


> Er no, That was entirely the point of my message.
> I was replying to your offer to change the sources of pdftex. If 

No I was offering to write the new code for XeTeX. Personally,
I do not care about pdfTeX! 

 
>that were done
> people would _not_ be able to use their ancient (or even not ancient)
> files and get the same results.



That is why they ship all these different programs!
If all programs had the same functionality, there would
be no need to include them in the same distribution.
You simple want XeTeX and others to follow what you
think is correct. I just said that it is good idea
to provide this functionality but we/they should implement
it the way we/they like. You want pseudo-random number, you
will get them. 


A.S.

--
Apostolos Syropoulos
Xanthi, Greece


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[David Carlisle]

On 14 November 2016 at 09:22, Apostolos Syropoulos
 wrote:
>>
>> Changing the source isn't the issue, are you offering also to help
>> with user support to manage the change
>> and explain to people why it's not possible to re-create documents
>> that they created before they updated
>> their tex system?
>>
>
> By following this conservative line of thought, we should
> not use OpenType fonts and stick to plain old but good Type 1
> fonts!

No, not at all. There is a big difference between offering new functionality
and offering existing functionality. when implementing existing tex features.

xetex (far more than luatex) follows tex very closely, so that in the
main, if you are
not using the extended features it works exactly like tex. All I am
suggesting is that
design philosophy should apply here.

> In fact, I would recommend dropping support for Type 1
> fonts from XeTeX. If you want Type 1 fonts, use the ancient
> engines.

>  Also, each TeX destribution comes with all these
> TeX-based engines. So these people can always use their
> ancient files with no problem!

Er no, That was entirely the point of my message.
I was replying to your offer to change the sources of pdftex. If that were done
people would _not_ be able to use their ancient (or even not ancient)
files and get the same results.

David


>
>
> A.S.
> --
> Apostolos Syropoulos
> Xanthi, Greece


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[David Carlisle]

On 13 November 2016 at 16:50, Apostolos Syropoulos
 wrote:

>
> OK please let me know how the pdftex thing is implemented and I will do the
> thing with drand48. Then people can check the implementation, etc.
>
> A.S.
>

People using a fixed seed to get reproducible results would not be
well served by a change in the random number generator. Just because
something would be easy to change doesn't mean that change is always a
good idea.

A document using a fixed seed would have been generating reproducible
results with pdftex since these functions were added more than ten
years ago.

Changing the source isn't the issue, are you offering also to help
with user support to manage the change
and explain to people why it's not possible to re-create documents
that they created before they updated
their tex system?

That said, if you want to look, the version in pdftex is

@* \[7b] Random numbers.

starting at line 2507 of  source/texk/web2c/pdftexdir/pdftex.web

Sometimes change is necessary but here it really isn't clear that the
statistical properties of drand48 are superior to Knuth's algorithm
from metafont  (I've never seen any comparison of these, only between
drand48 and C rand)
Or whether any of these algorithms have any real flaws when used with
sequence lengths of the sort realistically
practical in a tex document.

David


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Akira Kakuto]

Dear Joseph,


- \(pdf)uniformdeviate
- \(pdf)normaldeviate
- \(pdf)randomseed
- \(pdf)setrandomseed
(LuaTeX drops the 'pdf' part of the names.)


H. Kitagawa, the author of eptex, has added
\pdfuniformdeviate
\pdfnormaldeviate
\pdfrandomseed
\pdfsetrandomseed
in eptex and euptex (r42506 in TL).

Best,
Akira



--
Subscriptions, Archive, and List information, etc.:
 http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[David Carlisle]

On 13 November 2016 at 16:47, Apostolos Syropoulos
 wrote:
>>
>> That's true except that as pdftex and luatex have this already and have had
>> for years, in this particular instance it's playing catchup that is
>> being suggested.
>> Not promoting novel functionality.
>>
>
> So just because the design in pdftex and luatex was conservative, if not 
> flawed,
> it doesn't mean we should follow their "lead".

that is the model in much of xetex as far as features not relating to
unicode and font support are concerned:
that something that works in (pdf)tex and xetex should work the same
way in the two systems.

> The point here is to produce
> pseudo-random numbers. How they will be produced is something that the end 
> user
> should not worry about.

It depends. currently, if you set the seed you get the same numbers
from luatex and pdftex.
if xetex uses a different RNG then you would get different numbers in
xetex. that complicates some
things not least testing cross platform compatibility.



> In addition, it would be a nice opportunity for pdftex
> and luatex to "upgrade".

Perhaps. But that wasn't the current request.
>
>
>>
>> Not really. If the main use cases are met by the existing code then engine
>> differences are just extra work for the maintainers of the macro format
>> to hide to provide a consistent interface to the end user.
>>
>
> This is a fallacy! Just becauce LaTeX can process UTF-8 encoded input files,
> it doesn't mean XeTeX is useless!

That isn't what I said.

> And there is no extra code.

Oh there would be _less_ code probably as you'd use a system call rather than
implement the function inline, that isn't the issue.

> It would be just
> a simple system call. So instead of implementing random with a call to
>
>
> long random(void)
>
>
> one should implement it with a call to
>
>
> long lrand48(void)
>
> A.S.


David


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[David Carlisle]

On 13 November 2016 at 15:46, Apostolos Syropoulos
 wrote:

> Let me say that I find this point of view far too conservative. OK METAFONT
> lives in the previous century but luatex and xetex are products of this
> century so they must utilize what is better. Also, the point of introducing
> XeTeX was the need to handle things TeX had no provision.

That's true except that as pdftex and luatex have this already and have had
for years, in this particular instance it's playing catchup that is
being suggested.
Not promoting novel functionality.

>  Finally, it would
> an excellent opportunity for XeTeX to do something better that the others

Not really. If the main use cases are met by the existing code then engine
differences are just extra work for the maintainers of the macro format
to hide to provide a consistent interface to the end user.

>
> Regards,
>
> Apostolos
>

David


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Karljürgen Feuerherm]

Of course. And while the harvest might be greater (not necessarily great as 
such), the workers are few….

> On Nov 13, 2016, at 11:28, Joseph Wright  
> wrote:
> 
> On 13/11/2016 15:49, Karljürgen Feuerherm wrote:
>> Well… does it have to be either/or, anyhow? Taking Apostolos’ last point, 
>> why not have a switch?
>> 
>> It makes sense to allow for cross-platform compatibility, but there’s no 
>> reason to think that *nobody* would appreciate an improved algorithm….
>> 
>> K
> 
> I'm not *against* an improved approach, but the work needed for 'a new
> implementation in pdfTeX, LuaTeX, XeTeX, ...' is greater than 'add
> more-or-less the current implementation from pdfTeX and LuaTeX to XeTeX,
> ...'.
> 
> Joseph
> 
> 
> 
> 
> --
> Subscriptions, Archive, and List information, etc.:
>  http://tug.org/mailman/listinfo/xetex




--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Joseph Wright]

On 13/11/2016 15:49, Karljürgen Feuerherm wrote:
> Well… does it have to be either/or, anyhow? Taking Apostolos’ last point, why 
> not have a switch?
> 
> It makes sense to allow for cross-platform compatibility, but there’s no 
> reason to think that *nobody* would appreciate an improved algorithm….
> 
> K

I'm not *against* an improved approach, but the work needed for 'a new
implementation in pdfTeX, LuaTeX, XeTeX, ...' is greater than 'add
more-or-less the current implementation from pdfTeX and LuaTeX to XeTeX,
...'.

Joseph




--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Karljürgen Feuerherm]

Well… does it have to be either/or, anyhow? Taking Apostolos’ last point, why 
not have a switch?

It makes sense to allow for cross-platform compatibility, but there’s no reason 
to think that *nobody* would appreciate an improved algorithm….

K

> On Nov 13, 2016, at 10:46, Apostolos Syropoulos  wrote:
> 
>> 
>> A major aim of the original request was to bring the functionality of
>> the xetex engine into line with pdftex and luatex to help in
>> developing cross engine macro code. So there are arguments for using
>> the same code here unless the existing code is really broken and
>> luatex/metafont/pdftex should all change.
>> 
> 
> 
> Let me say that I find this point of view far too conservative. OK METAFONT
> lives in the previous century but luatex and xetex are products of this
> century so they must utilize what is better. Also, the pint of introducing
> XeTeX was the need to handle things TeX had no provision. Fianlly, it would
> an excellent opportunity for XeTeX to do something better that the others
> 
> Regards,
> 
> Apostolos
> 
> 
> --
> Apostolos Syropoulos
> Xanthi, Greece
> 
> 
> --
> Subscriptions, Archive, and List information, etc.:
>  http://tug.org/mailman/listinfo/xetex




--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Apostolos Syropoulos]

>
> A major aim of the original request was to bring the functionality of
> the xetex engine into line with pdftex and luatex to help in
> developing cross engine macro code. So there are arguments for using
> the same code here unless the existing code is really broken and
> luatex/metafont/pdftex should all change.
> 


Let me say that I find this point of view far too conservative. OK METAFONT
lives in the previous century but luatex and xetex are products of this
century so they must utilize what is better. Also, the pint of introducing
XeTeX was the need to handle things TeX had no provision. Fianlly, it would
an excellent opportunity for XeTeX to do something better that the others

Regards,

Apostolos


--
Apostolos Syropoulos
Xanthi, Greece


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[David Carlisle]

On 13 November 2016 at 13:58, Apostolos Syropoulos
 wrote:
>
>
> Thank you very much for your prompt response. I would say that
> this feature makes the typesetting engine more independent and
> this is definitely a good thing. Also, I think it should be
> implemented using the drand48 function as Neelson has already
> suggested.
>
>
> A.S.
>

the statistical properties of drand48 are almost certainly better but
metafont/pdftex/luatex all use the code from mf.web (or for example as
C in luatex sources arithemtic.w)

@ And now let's complete our collection of numeric utility routines
by considering random number generation.
\MF\ generates pseudo-random numbers with the additive scheme recommended
in Section 3.6 of {\sl The Art of Computer Programming};
...
@ Finally, a normal deviate with mean zero and unit standard deviation
can readily be obtained with the ratio method (Algorithm 3.4.1R in
{\sl The Art of Computer Programming\/}).


A major aim of the original request was to bring the functionality of
the xetex engine into line with pdftex and luatex to help in
developing cross engine macro code. So there are arguments for using
the same code here unless the existing code is really broken and
luatex/metafont/pdftex should all change.

David


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Joseph Wright]

On 13/11/2016 13:04, Apostolos Syropoulos wrote:
>> to track the seed). The usefulness of pseudo-random numbers has come up
>> a few times recently, and so we'd like to address this. (Expandable
>> floating point evaluation is pretty handy as an end user!)
>  
> 
> Can you please elaborate on the usefulness of pseudo-random numbers in
> a typesetting engine? I think it is a good thing to add features but
> those features should be added for some good reason.

Indeed.

I've seen a variety of use cases for pseudo-random values, in particular
two which come up reasonably often. The first is selecting entries from
a larger 'pool' of values, for example for creating test papers. ('Use 5
out of the 20 questions I've written, picked at random'.) The second,
probably more common, case is in creating figures (for example using
TikZ). Depending on what one is representing, an element of
(pseudo)randomness is useful.

These use cases are beyond what might call the 'classical' idea of what
TeX is for, in the sense one could do them (and other uses) by hand or
using another tool and import into TeX. However, the programmable nature
of TeX attracts use in these ways. One can generate pseudo-random
numbers at the macro level, most obviously in the pgf package, and this
facility is used by many people. (My own use case for random values, in
creating some figures, uses the pgf implementation.) However, the need
to track the seed value to allow a pseudo-random sequence of values
means that any macro-based implementation is necessarily non-expandable.
The experience of the team with \fp_eval:n, the expandable FPU of the
expl3 bundle, suggests that expandable calculations are useful. In that
context, it would be nice to be able to offer some random value
abilities. (As noted at the start of this thread, that is already
possible in pdfTeX and LuaTeX and we will likely add something to the
FPU which currently will work with those engines only.)

Joseph



--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex

Re: [XeTeX] Random number primitives

[Martin Schröder]

2016-11-12 22:45 GMT+01:00 Joseph Wright :
> lower-level pseudo-random number generator (I assume from C: there is
> very little actual code in the pdfTeX WEB source to implement these).

IIRC it came from MetaPost.

Best
   Martin


--
Subscriptions, Archive, and List information, etc.:
  http://tug.org/mailman/listinfo/xetex