Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-22 Thread Steve Ratcliffe
Hi Ticker > Problem is that resources/sort/cp65001.txt doesn't give ordering to > lots of characters; it looks like it covers only about 10,500 of the > 1,112,064 possible code-points. Many of these non-ordered characters > are being used by the names in the tile in question. I used the program

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-22 Thread Ticker Berkin
im Auftrag von > Gerd Petermann > Gesendet: Donnerstag, 21. Oktober 2021 15:48 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index > from unicode tiles > > Hi Ticker, > > I agree that the original code isn't cl

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-22 Thread Gerd Petermann
o I cannot analyse the content further. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Donnerstag, 21. Oktober 2021 15:48 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-21 Thread Gerd Petermann
ements or not. Gerd Von: Gerd Petermann Gesendet: Donnerstag, 21. Oktober 2021 11:21 An: Development list for mkgmap Betreff: AW: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi Ticker, so far I don't understand most of the chang

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-21 Thread Ticker Berkin
t: Mittwoch, 20. Oktober 2021 12:32 > An: Development list for mkgmap; Steve Ratcliffe > Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index > from unicode tiles > > Hi > > In the changes I've just made, I hope I've been consistent and fixed

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-21 Thread Gerd Petermann
eff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi In the changes I've just made, I hope I've been consistent and fixed all instances to use collator.compare() where scanning the results of a sort on the same table for a change. Also consistently setting strengt

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-20 Thread Ticker Berkin
Hi In the changes I've just made, I hope I've been consistent and fixed all instances to use collator.compare() where scanning the results of a sort on the same table for a change. Also consistently setting strength to SECONDARY (generally case-insensitive). There may be places where an indirect

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-20 Thread Ticker Berkin
Hi Gerd I didn't understand this either - Mdr29 with lowest refs to Mdr17, Mdr22, Mdr24, Mdr25 and Mdr26 is beyond me so I thought it best leave that part untouched. Ticker On Wed, 2021-10-20 at 07:59 +, Gerd Petermann wrote: > Hi Ticker, > > please double check Mdr25: > I just wonder why w

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-20 Thread Gerd Petermann
k and the classes in mkgmap. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Mittwoch, 20. Oktober 2021 09:59 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi Ticker

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-20 Thread Gerd Petermann
An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi Gerd Here it is Ticker On Tue, 2021-10-19 at 09:22 +, Gerd Petermann wrote: > Hi Ticker, > > yes, please remove all unrelated optimizations.

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-19 Thread Ticker Berkin
et: Dienstag, 19. Oktober 2021 11:03 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building > index from unicode tiles > > Hi Gerd > > I'd removed the change relating to clearing the reference to the Sort > object t

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-19 Thread Gerd Petermann
t; An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building > index from unicode tiles > > Hi Gerd > > Here is first version of the changes to improve MDR unicode and stop > the crash. > > It always provides a PRIMARY strength sort value,

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-19 Thread Ticker Berkin
ap > Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building > index from unicode tiles > > Hi Gerd > > Here is first version of the changes to improve MDR unicode and stop > the crash. > > It always provides a PRIMARY strength sort value, both in the key fo

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-19 Thread Gerd Petermann
1 16:36 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi Gerd Here is first version of the changes to improve MDR unicode and stop the crash. It always provides a PRIMARY strength sort value, both in the key for sorting and direct

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Ticker Berkin
Hi Gerd Here is first version of the changes to improve MDR unicode and stop the crash. It always provides a PRIMARY strength sort value, both in the key for sorting and direct comparison when using the collator. Previously neither of these would have anything for a unicode character not mentione

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Gerd Petermann
p-dev] java.lang.AssertionError while building index from unicode tiles Hi Gerd In imgfmt/app/srt/Sort.java around line 853: // Get the first non-ignorable at this level int c = chars[pos++ & 0xff]; if (!hasPage(c >>> 8)) { I'm at a loss to understand the

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Ticker Berkin
Hi Gerd In imgfmt/app/srt/Sort.java around line 853: // Get the first non-ignorable at this level int c = chars[pos++ & 0xff]; if (!hasPage(c >>> 8)) { I'm at a loss to understand the 0xff mask! am I missing something? Ticker _

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Ticker Berkin
e same > problem for other names like those for highways, POI etc? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 18. Oktober 2021 09:58 > An: Development list for mkgmap > Betreff: Re:

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Gerd Petermann
roblem for other names like those for highways, POI etc? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 18. Oktober 2021 09:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-18 Thread Ticker Berkin
Hi Although 2 16-bit items (surrogate pairs in UTF-16 speak) are required to represent many Chinese characters, this isn't the significant problem in this case. Problem is that resources/sort/cp65001.txt doesn't give ordering to lots of characters; it looks like it covers only about 10,500 of the

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-17 Thread Gerd Petermann
ávila Gesendet: Sonntag, 17. Oktober 2021 13:48 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles In that case, it seems estrange that only 2 of 67 tiles of China map cause problems, doesn't it? El 17/10/21 a las 12:

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-17 Thread Carlos Dávila
In that case, it seems estrange that only 2 of 67 tiles of China map cause problems, doesn't it? El 17/10/21 a las 12:16, Ticker Berkin escribió: Hi It is most likely that this problem is because Chinese requires 2 UTF16 chars to encode many of its characters - see https://softwareengineeri

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-17 Thread Ticker Berkin
Hi It is most likely that this problem is because Chinese requires 2 UTF16 chars to encode many of its characters - see https://softwareengineering.stackexchange.com/questions/102205/should-utf-16-be-considered-harmful I think it is only  --index processing where this is a problem mkgmap. I'll

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-15 Thread Ticker Berkin
Hi I can also reproduce this. I'll investigate, but am no expert on java sort/collation. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-15 Thread Gerd Petermann
ermann Gesendet: Freitag, 15. Oktober 2021 10:09 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles Hi Carlos, I can reproduce the crash. Not sure where to fix this yet... Gerd Von

Re: [mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-15 Thread Gerd Petermann
building index from unicode tiles Hi all I'm getting error below while building index from this tile: https://files.mkgmap.org.uk/download/523/31177029.o5m. Minimum mkgmap options triggering the error are: java -jar mkgmap-trunk.jar --bounds=bounds.zip --index --unicode 3117702

[mkgmap-dev] java.lang.AssertionError while building index from unicode tiles

2021-10-14 Thread Carlos Dávila
Hi all I'm getting error below while building index from this tile: https://files.mkgmap.org.uk/download/523/31177029.o5m. Minimum mkgmap options triggering the error are: java -jar mkgmap-trunk.jar --bounds=bounds.zip --index --unicode 31177029.o5m Another tile from the same splitter run al