Re: [mkgmap-dev] splitter r586 still doesn't always align properly with --align-for-dem option

[Gerd Petermann]

Hi Andrzej,

too sad, this all sounds very reasonable :(

Besides that I found only one very small problem in splitter:
It uses a slightly different method to convert degrees to map units compared to 
mkgmap.
I think the one used in mkgmap is good, the one in splitter is only close to 
correct.
mkgmap:
public static int toMapUnit(double l) {
double DELTA = 360.0D / (1 << 24) / 2; // Correct rounding
if (l > 0)
return (int) ((l + DELTA) * (1 << 24) / 360);
return (int) ((l - DELTA) * (1 << 24) / 360);
}
splitter uses nearly the same code but
double DELTA = 0.01; // TODO check if we really mean this

The value in mkgmap is 1.0728836059570312E-5 which is much closer to the exact 
value 0,107288360595703125

The real problem is what you already described:
A 32 bit DEM value that is far from 0,0 and perfectly aligned to 1/1200° 
degress is also perfectly aligned to 2^32 / (360 * 1200)  which is ~ 
9942,0539259259259259259259259259
but it is never aligned to 9942.

Gerd


Von: mkgmap-dev  im Auftrag von Andrzej 
Popowski 
Gesendet: Freitag, 19. Januar 2018 23:35:00
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter r586 still doesn't always align properly 
with --align-for-dem option

Hi Gerd,

 > What makes you think that 0,0 is used instead of the upper left cornen
 > given in the DEM section header(s)?

In Garmin's DEM coordinates are always a multiple of DEM pixel size
(dem-dists). The result is, that all DEMs data are aligned with 0;0. It
makes easy to join different tiles, since all tiles share the same
alignment.

Garmin uses DEM pixel size, which is not optimal for HGT. Looks like
they don't care about HGT alignment, while obviously they use SRTM data.

There should be no shifting of DEM, if DEM coordinates are strictly
respected by Mapsource od BaseCamp. But there could be artifacts on tile
borders.

Anyway, it is just a guess. We can make some test with Mapsource, but
would be difficult to test all different devices. That is why I would
prefer to make DEM exactly like Garmin, which means, that we can't get
good alignment with HGT.

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] addresses in the United States

[Greg Troxel]

(I have been using mkgmap for a really long time, and have been a little
out of touch with it the last few years mostly due to using OsmAnd on my
phone, although I still build maps for my Etrex 30 for hiking.)

I built osm.img from Geofabrik's us-northeast.  On my Nuvi 50, I find
that I can find Massachusetts towns, but not towns in Rhode Island.  I
realized this was due to my not really understanding the address options
and using them wrong/not-at-all.  So I read the docs about precompiled
bounds and sea and added those, using the files from mkgmap.org.uk, and
rebuilt.  Hence I am not asking about my earlier not-right builds.

With the new osm.img, in address search, there is only "United States"
vs "Canada", and no per-state option.  Towns can be found in any of the
states in my source file, but they show with their counties only, not
the state.  If you pick the right town/county combination, streets are
found, and I believe housenumbers, when they exist in OSM.  This is
harder than you might think because we basically reuse town names and
county names from England separately in each State :-)

When building, I first use splitter, and I think I have not had any
splitter issues in many years.  Then I run mkgmap (details at end).

Note that in addition to us-northeast.obf, I also add a file which has
parcel boundaries in my town convreted to OSM format, and I have a style
rule that shows them as marine depth contours.  This has been working
for many years.  My style diff in lines is:
  +# 0x23 is depth countour - thin.  Wacky but useful.  0x1c is too heavy
  +boundary=parcel [0x23 resolution 20]
I don't think this matters, but I don't feel right leaving out the info.

For no good reason, my recent build was done with
  r3997 | gerd | 2017-09-26 03:49:38 -0400 (Tue, 26 Sep 2017) | 1 line
and I am going to update and try again, without the --region-* arguments.
Other than the parcel style change my only other local changes are doc
fixes that I need to check and submit some day.

My questions are:

  1. I have leftover --region-* arguments in my script, and I'm
 wondering if that's the issue.  Should I remove those?  Does that
 explain the state/county confusion?

  2. I would expect the admin boundaries for state, county and town to
 be used from the bounds file, to fill in those address elements for
 roads, as well as for explicitly tagged addresses if they only have
 housenumber/street.  Is that right?   Fairly clearly the town and
 county boundaries are being used correctly.

 (Actually also county boundaries for counties that do not have
 admin_level=6 because of overzealous nonlocals who think it's
 helpful to remove those for counties that don't legally have a
 separate government, even though nobody who uses the map has any
 idea about this fine point and everyone refers to counties as a
 real thing.  But counties are showing up, so that's not the
 problem.)

  3. To make the best map possible for normal people to use navigating
 with a nuvi, without crossing into custom styles, should I be doing
 something different?

  4. Given the goal in (3), should I be crossing into a custom style?

  5. In addition to the admin hierarchy, which is what everyone thinks
 of in terms of addresses, there are also a lot of place names (as
 nodes) in OSM, with tags like village and hamlet.  These are names
 for where there were concentrations of house long ago, and many are
 in use today.  People use them as in "I live near Tracey's Corner"
 but would never put "Tracey's Corner" as a component of their
 address.  by using is_in, I think I'm not using those, and I don't
 think this is related to my problem anyway.

java \
-enableassertions \
-Xmx3072m \
-jar splitter.jar \
--output=pbf \
--max-nodes=${MAXNODES} \
--keep-complete=no \
$INPUT \

java \
-enableassertions \
-Xmx6144m \
-jar mkgmap.jar \
--max-jobs \
--latin1 \
--description="OSM_gdt" \
--country-abbr="US" \
--country-name="United States" \
--region-abbr="MA" \
--region-name="Massachusetts" \
--family-id=1500 \
--family-name="OSM_gdt_family" \
--product-id=13 \
--series-name="OSM_gdt_series" \
--area-name="OSM_gdt_area" \
--draw-priority=60 \
--overview-mapname=6324 \
--tdbfile \
--precomp-sea=sea.zip \
--reduce-point-density=4 \
--reduce-point-density-polygon=8 \
--merge-lines \
--route \
--check-roundabouts \
--check-roundabout-flares \
--remove-short-arcs \
--adjust-turn-headings \
--report-similar-arcs \
--report-dead-ends=2 \
--add-pois-to-areas \
--check-styles \
--index \
--bounds=bounds.zip \
--housenumbers \
--location-autofill=is_in \
-c template.args \
--gmapsupp


Thanks,
Greg (osm user gdt)


signature.asc
Description: PGP signature

Re: [mkgmap-dev] mkgmap r4048 seems to fix problem with hill shading

[Andrzej Popowski]

Hi,

I still observe the first problem with small rectangles without shading. 
But I see it only when displaying small islands on a see. Like for 
example Príncipe Island or Ascension Island.


Could it be, that processing sea level or voids around island could 
cause similar effects like polygon border?


All my maps are currently compiled with patched mkgmap, I can't tell if 
original mkgmap shows this effect too.


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] HGT - getElevation()

[Andrzej Popowski]

Hi,

quick comparison between r4068 original and r4070 with bicubic 
interpolation. Both use pbf aligned to HGT and the same 3 arc degree HGT 
with many voids.


http://files.mkgmap.org.uk/download/401/orig-aligned.png
http://files.mkgmap.org.uk/download/402/bicu-aligned.png

DEM shading moved again. DEM heights are similar on both maps, there is 
difference like 20-30m between DEM peaks.


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] HGT - getElevation()

[Andrzej Popowski]

Hi Gerd,

I have included bicubic interpolation created by Ken Perlin:
https://github.com/mlevans/pretty-heatmaps/blob/master/DensityArrayCreation/src/DensityArrayCreation/Bicubic.java

Bicubic interpolation is used only for layers with good resolution (3 
arc seconds or better). For other cases bilinear interpolation is used.


Patch is attached.

--
Best regards,
Andrzej

Index: src/uk/me/parabola/imgfmt/app/dem/DEMFile.java
===
--- src/uk/me/parabola/imgfmt/app/dem/DEMFile.java  (revision 4070)
+++ src/uk/me/parabola/imgfmt/app/dem/DEMFile.java  (working copy)
@@ -52,20 +52,57 @@
 * @param outsidePolygonHeight the height value that should be used for 
points outside of the bounding polygon 
 */
public void calc(Area area, java.awt.geom.Area demPolygonMapUnits, 
String pathToHGT, List pointDistances, short outsidePolygonHeight) {
-   HGTConverter hgtConverter = new HGTConverter(pathToHGT, area, 
demPolygonMapUnits);
+   // HGT area is extended by 0.01 degree in each direction
+   HGTConverter hgtConverter = new HGTConverter(pathToHGT, area, 
demPolygonMapUnits, 0.01D);
hgtConverter.setOutsidePolygonHeight(outsidePolygonHeight);
 
+   int top = area.getMaxLat() * 256;
+   int bottom = area.getMinLat() * 256;
+   int left = area.getMinLong() * 256;
+   int right = area.getMaxLong() * 256;
+
int zoom = 0;
int lastDist = pointDistances.get(pointDistances.size()-1); 
for (int pointDist : pointDistances) {
-   DEMSection section = new DEMSection(zoom++, area, 
hgtConverter, pointDist, pointDist == lastDist);
+   int distance = pointDist;
+   if (distance == -1) {
+   int res = (hgtConverter.getHighestRes() > 0) ? 
hgtConverter.getHighestRes() : 1200;
+   distance = (int) ((1 << 29) / (res * 45));
+   }
+   // last 4 bits of distance should be 0
+   distance = ((distance + 8)/16)*16;
+
+   int xtop = top;
+   int xleft = left;
+
+   // align DEM to distance raster, if distance not bigger 
than widening of HGT area
+   if (distance < (int)Math.floor((0.01D/45.0D * (1 << 
29 {
+   if (xtop >= 0) {
+   xtop -= xtop % distance;
+   if (xtop < 0x3FFF - distance)
+   xtop += distance;
+   }
+   else {
+   xtop -= xtop % distance;
+   }
+
+   if (xleft >= 0) {
+   xleft -= xleft % distance;
+   }
+   else {
+   xleft -= xleft % distance;
+   if (xleft > Integer.MIN_VALUE + 
distance)
+   xleft -= distance;
+   }
+   }
+
+   // to improve: DEM secition should get area in 32bit 
units
+   DEMSection section = new DEMSection(zoom++, xtop, 
xleft, xtop - bottom, right - xleft, hgtConverter, distance, pointDist == 
lastDist);
demHeader.addSection(section);
}
return;
}
 
-   
-   
public void write() {
ImgFileWriter w = getWriter();
if (w instanceof BufferedImgFileWriter) {
Index: src/uk/me/parabola/imgfmt/app/dem/DEMSection.java
===
--- src/uk/me/parabola/imgfmt/app/dem/DEMSection.java   (revision 4070)
+++ src/uk/me/parabola/imgfmt/app/dem/DEMSection.java   (working copy)
@@ -48,16 +48,12 @@
private int maxHeight = Integer.MIN_VALUE;
private List tiles = new ArrayList<>();

-   public DEMSection(int zoomLevel, Area bbox, HGTConverter hgtConverter, 
int pointDist, boolean lastLevel) {
+   public DEMSection(int zoomLevel, int areaTop, int areaLeft, int 
areaHeight, int areaWidth, HGTConverter hgtConverter, int pointDist, boolean 
lastLevel) {
this.zoomLevel = zoomLevel;
this.lastLevel = lastLevel;

-   if (pointDist == -1) {
-   int res = (hgtConverter.getHighestRes() > 0) ? 
hgtConverter.getHighestRes() : 1200;
-   pointDist = (int) ((1 << 29) / (res * 45));
-   }
-   this.top = bbox.getMaxLat() * 256;
-   

Re: [mkgmap-dev] splitter r586 still doesn't always align properly with --align-for-dem option

[Andrzej Popowski]

Hi Gerd,

> What makes you think that 0,0 is used instead of the upper left cornen
> given in the DEM section header(s)?

In Garmin's DEM coordinates are always a multiple of DEM pixel size 
(dem-dists). The result is, that all DEMs data are aligned with 0;0. It 
makes easy to join different tiles, since all tiles share the same 
alignment.


Garmin uses DEM pixel size, which is not optimal for HGT. Looks like 
they don't care about HGT alignment, while obviously they use SRTM data.


There should be no shifting of DEM, if DEM coordinates are strictly 
respected by Mapsource od BaseCamp. But there could be artifacts on tile 
borders.


Anyway, it is just a guess. We can make some test with Mapsource, but 
would be difficult to test all different devices. That is why I would 
prefer to make DEM exactly like Garmin, which means, that we can't get 
good alignment with HGT.


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r586 still doesn't always align properly with --align-for-dem option

[Gerd Petermann]

Hi Andrzej,

I agree that it makes no sense to align to hgt when Garmin uses 0,0 as base
and data is far away from this point.
Probably it is a good idea to think about the way how the Garmin software can
determine what data it has to read to render a given rectangle. What makes you 
think
that 0,0 is used instead of the upper left cornen given in the DEM section 
header(s)?

Gerd


Von: mkgmap-dev  im Auftrag von Andrzej 
Popowski 
Gesendet: Freitag, 19. Januar 2018 17:06:42
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter r586 still doesn't always align properly 
with --align-for-dem option

Hi Gerd,

my current opinion is, that aligning to HGT won't work. The way Garmin
sets coordinates in DEM suggests, that all DEM is aligned to coordinates
0;0. Since pixel distance for DEM can be set exactly as in HGT, the
farther form 0;0 the bigger misalignment. Then at some point could
appear a shift of DEM, like we have already observed for unaligned
versions. I estimate, that for dem-dists=9942 this could be above 80 degree.

--
Best regards,
Andrzej

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] HGT - getElevation()

[Andrzej Popowski]

Hi Gerd,

> I'd prefer to use unmodified dem-dist values for now, besides that the
> unit tests don't compile with it.

I'm trying to get DEM looking like in Garmins img. The actual rounding 
in code can be disabled, but then it would be up to user, to set correct 
distances. I don't know, if rounding is important. Maybe some devices 
need it?


> After adding 0.01 as new parm some tests fail.

This looked like an easy way to get HGT without computing and comparing 
values for all layers. It could be done better, if the idea fo this 
patch worked out.


Sorry about tests, I don't know how to execute them.

Now I consider to add bicubic interpolation, like this code:
https://github.com/mlevans/pretty-heatmaps/blob/master/DensityArrayCreation/src/DensityArrayCreation/Bicubic.java

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r586 still doesn't always align properly with --align-for-dem option

[Andrzej Popowski]

Hi Gerd,

my current opinion is, that aligning to HGT won't work. The way Garmin 
sets coordinates in DEM suggests, that all DEM is aligned to coordinates 
0;0. Since pixel distance for DEM can be set exactly as in HGT, the 
farther form 0;0 the bigger misalignment. Then at some point could 
appear a shift of DEM, like we have already observed for unaligned 
versions. I estimate, that for dem-dists=9942 this could be above 80 degree.


--
Best regards,
Andrzej

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] splitter r586 still doesn't always align properly with --align-for-dem option

[Gerd Petermann]

Hi all,

I don't know yet why but alignment doesn't always work well.
One problem is that we keep bbox data only with 24 bit resolution, but there 
must be more to it.

I'll try to find out more but I have not much time until monday.

Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev