Hi Gerd
Patch attached that handles UTF16 surrogate pairs so that mkgmap style
filters substring and highway-shield give correct results, the LBL
trimming leaves the correct amount and neither end up with 1/2 a
character.
I've looked through all the substring operations and I don't think any
others will cause problems. There might just be a few str.length() and
str.charAt() or other indexing that might need attention but this would
require a lot more searching. I've left the handling for
MALFORMED_INPUT so these shouldn't matter.
Ticker
On Wed, 2021-12-29 at 09:16 +0000, Ticker Berkin wrote:
> Hi Gerd
>
> I'll look at this sometime. I while ago I found something in one of
> the
> MDR sections (probably the short strings) that handled something like
> this.
>
> Ticker
>
>
> On Tue, 2021-12-28 at 13:22 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > okay, maybe you find time to implement a better solution.
> > I've committed the patch with r4838.
> >
> > @Arndt: Please check if this solves the problems with your maps.
> > Not sure about this because you said mkgmap stops without any
> > reaction
> > while this problem was a loop.
> >
> > Gerd
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Index: src/uk/me/parabola/imgfmt/app/lbl/LBLFile.java
===================================================================
--- src/uk/me/parabola/imgfmt/app/lbl/LBLFile.java (revision 4838)
+++ src/uk/me/parabola/imgfmt/app/lbl/LBLFile.java (working copy)
@@ -61,7 +61,7 @@
private static final int OFFSET_MULTIPLIER = 1;
/** Garmin software doesn't display longer labels */
- private static final int MAX_LABEL_LEN = 170;
+ private static final int MAX_LABEL_LEN = 170;
public LBLFile(ImgChannel chan, Sort sort) {
this.sort = sort;
@@ -144,17 +144,24 @@
* @return A reference to the created label
*/
public Label newLabel(String text, boolean cutToMaxLen) {
- if (text != null && cutToMaxLen) {
- int trimmedLen = text.length();
- if (trimmedLen > MAX_LABEL_LEN) {
- trimmedLen = MAX_LABEL_LEN;
- while (trimmedLen > 0 && text.charAt(trimmedLen) == ' ')
- trimmedLen--;
+ if (text != null && cutToMaxLen && text.length() > MAX_LABEL_LEN) {
+ int trimmedLen;
+ try {
+ trimmedLen = text.offsetByCodePoints(0, MAX_LABEL_LEN);
+ } catch (IndexOutOfBoundsException e) {
+ trimmedLen = 0; // short enough
+ }
+ if (trimmedLen > 0) {
+ do { // logic elsewhere removes leading, trailing and multiple spaces, but now might have trailing; remove multiple anyway
+ --trimmedLen;
+ if (text.charAt(trimmedLen) != ' ')
+ break;
+ } while (trimmedLen > 0);
text = text.substring(0, trimmedLen + 1);
}
}
EncodedText encodedText = textEncoder.encodeText(text);
-
+
Label l = labelCache.get(encodedText);
if (l == null) {
l = new Label(encodedText.getChars());
Index: src/uk/me/parabola/mkgmap/osmstyle/actions/HighwaySymbolFilter.java
===================================================================
--- src/uk/me/parabola/mkgmap/osmstyle/actions/HighwaySymbolFilter.java (revision 4838)
+++ src/uk/me/parabola/mkgmap/osmstyle/actions/HighwaySymbolFilter.java (working copy)
@@ -85,11 +85,14 @@
for (char c : shieldText.toCharArray()) {
if (Character.isDigit(c)) {
isAlphaNum = true; // Consider alphanumeric if we find one or more digits
+ break;
}
}
// Check if shield exceeds maximum length:
- if ( (isAlphaNum && shieldText.length() > maxAlphaNum) || (! isAlphaNum) && shieldText.length() > maxAlpha ) {
+ int codePointLength = shieldText.codePointCount(0, shieldText.length());
+ if (isAlphaNum && codePointLength > maxAlphaNum || !isAlphaNum && codePointLength > maxAlpha) {
+ // ??? this doesn't do as described in the documentation
return value; // If so, return original value
} else {
return prefix + shieldText; // If not, return condensed value with magic code
Index: src/uk/me/parabola/mkgmap/osmstyle/actions/SubstringFilter.java
===================================================================
--- src/uk/me/parabola/mkgmap/osmstyle/actions/SubstringFilter.java (revision 4838)
+++ src/uk/me/parabola/mkgmap/osmstyle/actions/SubstringFilter.java (working copy)
@@ -56,14 +56,28 @@
}
protected String doFilter(String value, Element el) {
- if (value == null) return null;
+ if (value == null)
+ return null;
if (start > value.length())
return null;
+ int codePointStart = 0;
+ if (start > 0)
+ try {
+ codePointStart = value.offsetByCodePoints(0, start);
+ } catch (IndexOutOfBoundsException e) {
+ return null;
+ }
if (args == 1 || end > value.length()) {
- return value.substring(start);
+ return value.substring(codePointStart);
}
if (args == 2) {
- return value.substring(start, end);
+ int codePointEnd = 0;
+ try {
+ codePointEnd = value.offsetByCodePoints(start, end-start);
+ } catch (IndexOutOfBoundsException e) {
+ return value.substring(codePointStart);
+ }
+ return value.substring(codePointStart, codePointEnd);
}
return value;
}
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
