[XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Peter Mukunda Pasedach
Dear XeTeX list, I am dealing with a collection of texts in Sanskrit, for which the builtin limitation of TeX to not perform hyphenation after the 63rd character of a string is imposing a serious limitation, as such strings do occur. One reason for this is that one can freely form very long compou

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Jonathan Kew
On 15/3/16 14:24, Peter Mukunda Pasedach wrote: Dear XeTeX list, I am dealing with a collection of texts in Sanskrit, for which the builtin limitation of TeX to not perform hyphenation after the 63rd character of a string is imposing a serious limitation, as such strings do occur. One reason for

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Peter Mukunda Pasedach
Dear Jonathan, yes, recompiling xetex is fine! At 255 characters I still have 32 occurences left, at 500 two, and at 1000 zero. Thanks for looking into this! Peter On Tue, Mar 15, 2016 at 3:46 PM, Jonathan Kew wrote: > On 15/3/16 14:24, Peter Mukunda Pasedach wrote: >> >> Dear XeTeX list, >> >

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Doug McKenna
r wraparound in the arithmetic. FWIW, Doug McKenna - Original Message - From: "Peter Mukunda Pasedach" To: "XeTeX (Unicode-based TeX) discussion." Sent: Tuesday, March 15, 2016 9:13:08 AM Subject: Re: [XeTeX] Hyphenation of strings of more than 63 characters Dear

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Jonathan Kew
On 15/3/16 18:04, Doug McKenna wrote: There could be some subtle problems that simply changing the character count constant causes. In particular, the allocation size of a "whatsit" language node might also need changing, which would require adjusting other code in the core engine that assumes

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-15 Thread Zdenek Wagner
2016-03-16 0:06 GMT+01:00 Jonathan Kew : > On 15/3/16 18:04, Doug McKenna wrote: > >> There could be some subtle problems that simply changing the character >> count constant causes. >> >> In particular, the allocation size of a "whatsit" language node might >> also need changing, which would requ

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Philip Taylor
Jonathan Kew wrote: > I'll try to get an experimental patch ready shortly. Or, of course, > someone else is welcome to try. I don't think it's very hard, but it is > more than just a single number. I have never attempted to modify *TeX since it ceased to be compiled as Pascal (a language which

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Peter Mukunda Pasedach
I eventually managed to build xetex (on Ubuntu 14.04, with a TeX Live 2015 installation kept up to date) by cloning the xetex sources from sourceforge, then modified, as an experiment, xetex.web, replacing 63 by 1023 in the places that seemed relevant, only guessing which ones they were, noticed th

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Jonathan Kew
On 18/3/16 08:43, Peter Mukunda Pasedach wrote: I eventually managed to build xetex (on Ubuntu 14.04, with a TeX Live 2015 installation kept up to date) by cloning the xetex sources from sourceforge, then modified, as an experiment, xetex.web, replacing 63 by 1023 in the places that seemed releva

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Jonathan Kew
Peter, I have just pushed a new experimental branch named "max-hyph-len" to the xetex repository. The changes here implement a new integer parameter \XeTeXhyphenatablelength which defaults to 63 (to maintain existing behavior), but can be increased to allow longer words to be hyphenated. (

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Peter Mukunda Pasedach
Any news on this? If it's just one constant whose value I would have to increase in my private copy of the code, before recompling, for testing purposes, which one would that be? Peter On Wed, Mar 16, 2016 at 12:37 AM, Zdenek Wagner wrote: > 2016-03-16 0:06 GMT+01:00 Jonathan Kew : >> >> On 15/3

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Peter Mukunda Pasedach
I finally managed to build it in such a way that it identifies itself with the right version number, I had to do a new clone of the sources for that, didn't find out how to clean them up. Now how do I set the parameter, as \XeTeXhyphenatablelength=1023 somewhere in my preamble or as a command line

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Philip Taylor
Peter Mukunda Pasedach wrote: > I finally managed to build it in such a way that it identifies itself > with the right version number, I had to do a new clone of the sources > for that, didn't find out how to clean them up. Now how do I set the > parameter, as \XeTeXhyphenatablelength=1023 somew

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Peter Mukunda Pasedach
That would be fmtutil --all? Yes I ran that. Maybe something wrong with my test file? -- \documentclass[12pt]{article} \usepackage{fontspec} \usepackage{polyglossia} \setdefaultlanguage{sanskrit} \newfontfamily{\sanskritfont}{Latin Modern Roman} \XeTeXhyphenatablelength=1023 \begin{docume

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Philip Taylor
Peter Mukunda Pasedach wrote: > That would be fmtutil --all? Yes I ran that. Maybe something wrong > with my test file? Ah, there I can't help, I am afraid; it is written in LaTeX, a language with which I have near-zero familiarity. Is it possible for you to test in a plain e-TeX environment,

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Jonathan Kew
On 19/3/16 20:30, Peter Mukunda Pasedach wrote: That would be fmtutil --all? Yes I ran that. Maybe something wrong with my test file? Yes, in a sense... you're running into another quirk of TeX: it doesn't attempt to hyphenate the first word of a paragraph. And in your file, every word is the

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Jonathan Kew
On 17/3/16 05:16, Peter Mukunda Pasedach wrote: Any news on this? If it's just one constant whose value I would have to increase in my private copy of the code, before recompling, for testing purposes, which one would that be? Unfortunately, it's not just a single constant; there are a number o

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Peter Mukunda Pasedach
This seems to work, at least for my test file. I tried on one of my much more complex real files, including reledmac etc., and there your quick hack for first word of paragraph didn't work yet, but if I manually add the \hskip 0pt \relax at the beginning of a paragraph then it works there, too. For

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Zdenek Wagner
Remember that LaTeX macros may redefine \everypar. For instance all sectioning command (\section, \subsection etc.) are implemented via \@startsection which ends with \@afterheading which uses \everypar to change some penalties and indentation of the first paragraph. I am not sure whether it is res

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Jonathan Kew
On 19/3/16 21:57, Peter Mukunda Pasedach wrote: This seems to work, at least for my test file. I tried on one of my much more complex real files, including reledmac etc., and there your quick hack for first word of paragraph didn't work yet, That doesn't surprise me; I'm sure LaTeX will be maki

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-19 Thread Akira Kakuto
I have just pushed a new experimental branch named "max-hyph-len" to the xetex repository. The changes here implement a new integer parameter \XeTeXhyphenatablelength Users of windows can test xetex-0.6 by obtaining a 32bit binary: http://members2.jcom.home.ne.jp/wt1357ak/xetex-6-w3

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-20 Thread Philip Taylor
Working fine here, Akira-san, in conjunction with TeX Live 2014 using a slightly modified version of the Plain e-TeX test source suggested to Peter Mukunda Pasedach yesterday : > \uselanguage {sanskrit} > > \font \mainfont = "Latin Modern Roman" > \mainfont > \XeTeXhyphenatablelength = 1023 > \ra

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-20 Thread Melroch
Might the everyhook package be useful here? /bpj Den 19 mar 2016 23:15 skrev "Jonathan Kew" : > On 19/3/16 21:57, Peter Mukunda Pasedach wrote: > >> This seems to work, at least for my test file. I tried on one of my >> much more complex real files, including reledmac etc., and there your >> quic

Re: [XeTeX] Hyphenation of strings of more than 63 characters

2016-03-20 Thread Jonathan Kew
On 19/3/16 23:08, Akira Kakuto wrote: I have just pushed a new experimental branch named "max-hyph-len" to the xetex repository. The changes here implement a new integer parameter \XeTeXhyphenatablelength Users of windows can test xetex-0.6 by obtaining a 32bit binary: http://members2.j