Re: [mkgmap-dev] HGT - getElevation()

2018-01-18 Thread Gerd Petermann 
Hi Andrzej,

I'd prefer to use unmodified dem-dist values for now, besides that the unit 
tests don't compile with it.
After adding 0.01 as new parm some tests fail.
Probably not so important, as the unit test HGTConverterTest is bad anyway, it 
tests non-public methods
and your patch shows that this is a bad idea.

Gerd


Von: mkgmap-dev  im Auftrag von Andrzej 
Popowski 
Gesendet: Freitag, 19. Januar 2018 02:59:14
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] HGT - getElevation()

Hi Gerd,

I have prepared patch according to these guessed rules. It rounds
dem-dist and shift coordinates of top-left corner of DEM areas. Since
dem-dist doesn't correspond exactly to HGT, interpolation is always active.

I think there could be 2 ways to improve DEM quality. Interpolation
could be changed to bicubic or mkgmap could use preprocessed data with
correct pixel size. In the latter case probably data format other than
HGT should be used.

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] HGT - getElevation()

2018-01-18 Thread Andrzej Popowski 

Hi Gerd,

I have prepared patch according to these guessed rules. It rounds 
dem-dist and shift coordinates of top-left corner of DEM areas. Since 
dem-dist doesn't correspond exactly to HGT, interpolation is always active.


I think there could be 2 ways to improve DEM quality. Interpolation 
could be changed to bicubic or mkgmap could use preprocessed data with 
correct pixel size. In the latter case probably data format other than 
HGT should be used.


--
Best regards,
Andrzej
Index: src/uk/me/parabola/imgfmt/app/dem/DEMFile.java
===
--- src/uk/me/parabola/imgfmt/app/dem/DEMFile.java  (revision 4070)
+++ src/uk/me/parabola/imgfmt/app/dem/DEMFile.java  (working copy)
@@ -52,20 +52,57 @@
 * @param outsidePolygonHeight the height value that should be used for 
points outside of the bounding polygon 
 */
public void calc(Area area, java.awt.geom.Area demPolygonMapUnits, 
String pathToHGT, List pointDistances, short outsidePolygonHeight) {
-   HGTConverter hgtConverter = new HGTConverter(pathToHGT, area, 
demPolygonMapUnits);
+   // HGT area is extended by 0.01 degree in each direction
+   HGTConverter hgtConverter = new HGTConverter(pathToHGT, area, 
demPolygonMapUnits, 0.01D);
hgtConverter.setOutsidePolygonHeight(outsidePolygonHeight);
 
+   int top = area.getMaxLat() * 256;
+   int bottom = area.getMinLat() * 256;
+   int left = area.getMinLong() * 256;
+   int right = area.getMaxLong() * 256;
+
int zoom = 0;
int lastDist = pointDistances.get(pointDistances.size()-1); 
for (int pointDist : pointDistances) {
-   DEMSection section = new DEMSection(zoom++, area, 
hgtConverter, pointDist, pointDist == lastDist);
+   int distance = pointDist;
+   if (distance == -1) {
+   int res = (hgtConverter.getHighestRes() > 0) ? 
hgtConverter.getHighestRes() : 1200;
+   distance = (int) ((1 << 29) / (res * 45));
+   }
+   // last 4 bits of distance should be 0
+   distance = ((distance + 8)/16)*16;
+
+   int xtop = top;
+   int xleft = left;
+
+   // align DEM to distance raster, if distance not bigger 
than widening of HGT area
+   if (distance < (int)Math.floor((0.01D/45.0D * (1 << 
29 {
+   if (xtop >= 0) {
+   xtop -= xtop % distance;
+   if (xtop < 0x3FFF - distance)
+   xtop += distance;
+   }
+   else {
+   xtop -= xtop % distance;
+   }
+
+   if (xleft >= 0) {
+   xleft -= xleft % distance;
+   }
+   else {
+   xleft -= xleft % distance;
+   if (xleft > Integer.MIN_VALUE + 
distance)
+   xleft -= distance;
+   }
+   }
+
+   // to improve: DEM secition should get area in 32bit 
units
+   DEMSection section = new DEMSection(zoom++, xtop, 
xleft, xtop - bottom, right - xleft, hgtConverter, distance, pointDist == 
lastDist);
demHeader.addSection(section);
}
return;
}
 
-   
-   
public void write() {
ImgFileWriter w = getWriter();
if (w instanceof BufferedImgFileWriter) {
Index: src/uk/me/parabola/imgfmt/app/dem/DEMSection.java
===
--- src/uk/me/parabola/imgfmt/app/dem/DEMSection.java   (revision 4070)
+++ src/uk/me/parabola/imgfmt/app/dem/DEMSection.java   (working copy)
@@ -48,16 +48,12 @@
private int maxHeight = Integer.MIN_VALUE;
private List tiles = new ArrayList<>();

-   public DEMSection(int zoomLevel, Area bbox, HGTConverter hgtConverter, 
int pointDist, boolean lastLevel) {
+   public DEMSection(int zoomLevel, int areaTop, int areaLeft, int 
areaHeight, int areaWidth, HGTConverter hgtConverter, int pointDist, boolean 
lastLevel) {
this.zoomLevel = zoomLevel;
this.lastLevel = lastLevel;

-   if (pointDist == -1) {
-   int res = (hgtConverter.getHighestRes() > 0) ? 
hgtConverter.getHighestRes() : 1200;
-   pointDist = (int) ((1

Re: [mkgmap-dev] HGT - getElevation()

2018-01-18 Thread Andrzej Popowski 

Hi Gerd,

I have executed your script and looked at maps. I have tested them with 
artificial HGT too. Only the original-unaligned map has DEM at wrong 
position. It is moved by about 0.00033 degree right and down.


I looked at offset calculated as (DEM coordinate)%(dem-dist) and only 
for this map, offset was bigger than half of dem-dist. Any attempt to 
align HGT with data reduce this offset, but it still remain quite big, 
like 1300-3600, while dem-dist is 9942.


My guess is, that Mapsource/BaseCamp calculate DEM pixel positions 
globally, basing on dem-dist. If coordinates of DEM layers aren't 
aligned with with dem-dist and misalignment is too big, than Mapsource 
can fetch wrong DEM values.


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Gerd Petermann 
Hi Henning,

please try r586, see
http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=586

Gerd


Von: mkgmap-dev  im Auftrag von Henning 
Scholland 
Gesendet: Donnerstag, 18. Januar 2018 16:31:29
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

Hi Gerd
I imported both kml-files to josm and added some taggs to make it more
clear.

landuse=grass -> split result based on r584 and also roughly same as
polygon used by splitter

landuse=industrial -> result with r585 and alignment to DEM. These tiles
should be in templates args

landuse =industrial + natural=water -> these tiles r585 adds also to
templates.args, but they are mainly out of map area. Maybe only inside
with a small area because of new alignment.

Henning
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Henning Scholland 
Hi Gerd
I imported both kml-files to josm and added some taggs to make it more
clear.

landuse=grass -> split result based on r584 and also roughly same as
polygon used by splitter

landuse=industrial -> result with r585 and alignment to DEM. These tiles
should be in templates args

landuse =industrial + natural=water -> these tiles r585 adds also to
templates.args, but they are mainly out of map area. Maybe only inside
with a small area because of new alignment.

Henning

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Gerd Petermann 
Hi Henning,

yes, I also see such differences. Looking at it now...

Gerd


Von: mkgmap-dev  im Auftrag von Henning 
Scholland 
Gesendet: Donnerstag, 18. Januar 2018 16:28:16
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

Hi Gerd
I imported both kml-files to josm and added some taggs to make it more
clear.

landuse=grass -> split result based on r584 and also roughly same as
polygon used by splitter

landuse=industrial -> result with r585 and alignment to DEM. These tiles
should be in templates args

landuse =industrial + natural=water -> these tiles r585 adds also to
templates.args, but they are mainly out of map area. Maybe only inside
with a small area because of new alignment.

Henning
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Henning Scholland 
Hi Gerd
I imported both kml-files to josm and added some taggs to make it more
clear.

landuse=grass -> split result based on r584 and also roughly same as
polygon used by splitter

landuse=industrial -> result with r585 and alignment to DEM. These tiles
should be in templates args

landuse =industrial + natural=water -> these tiles r585 adds also to
templates.args, but they are mainly out of map area. Maybe only inside
with a small area because of new alignment.

Henning

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Henning Scholland 
Sorry, I forgotten to add this:
It seems the reason is that --polygon-desc-file is not working well any
more. So main reason is, that the map is getting bigger. You can see it
in last mails attached kml-files.

Henning

On 18.01.2018 22:33, Henning Scholland wrote:
> Hi Gerd,
> I have quite bad news.
>
> My map of China increased size about 300 mb (before 1.6 Gb, now 1.9Gb)
> just by following this suggestion adding --align-for-dem=1200.
>
> hgt-files are same, osm-data is same. bounding polygon for splitter is
> same and also tiles look similar. I was using latest mkgmap (4070) for
> both tests and
> x-dem-dist=9942,9942,9942,19884,39768
> x-overview-dem-dist=79536
>
> South east asia increased from 1.15 to 1.4 Gb with same work flow.
>
> I'm sorry :-(. If you need anything or if I should test something,
> please let me know.
>
> Henning
>
> On 18.01.2018 01:42, Gerd Petermann wrote:
>> Hi all,
>>
>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=585
>>
>> If I got that right you can use dem-dist 9942 for level 0 if hgt files are 
>> in 3'' resolution and splitter
>> was executed with --align-for-dem=1200.
>> For hgt 1'' files use dem-dist 3342 for level 0 and --align-for-dem=3600.
>>
>> This should reduce DEM size a lot AND increase precision.
>>
>> Gerd
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

2018-01-18 Thread Gerd Petermann 
Hi Henning,

this is not surprising for me. Tiles should be a bit different, if you look at 
them in JOSM you have to zoom to maybe 30 m.

Why does the size grow?
The effect of the alignment is higher precision, means, less interpolation. 
With interpolation, the values are averaged, so
deltas are smaller and thus the number of bits to store the deltas are smaller.
So, if you already used 9942 as dem-dist for level 0 before the only 
improvement is precision.
Other users use e.g. dem-dists=3312,3312,... and for those I would also expect 
a decrease in size.

OK?
Gerd


Von: mkgmap-dev  im Auftrag von Henning 
Scholland 
Gesendet: Donnerstag, 18. Januar 2018 15:33:56
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter r585 implements new option align-for-dem

Hi Gerd,
I have quite bad news.

My map of China increased size about 300 mb (before 1.6 Gb, now 1.9Gb)
just by following this suggestion adding --align-for-dem=1200.

hgt-files are same, osm-data is same. bounding polygon for splitter is
same and also tiles look similar. I was using latest mkgmap (4070) for
both tests and
x-dem-dist=9942,9942,9942,19884,39768
x-overview-dem-dist=79536

South east asia increased from 1.15 to 1.4 Gb with same work flow.

I'm sorry :-(. If you need anything or if I should test something,
please let me know.

Henning

On 18.01.2018 01:42, Gerd Petermann wrote:
> Hi all,
>
> see http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=585
>
> If I got that right you can use dem-dist 9942 for level 0 if hgt files are in 
> 3'' resolution and splitter
> was executed with --align-for-dem=1200.
> For hgt 1'' files use dem-dist 3342 for level 0 and --align-for-dem=3600.
>
> This should reduce DEM size a lot AND increase precision.
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] r4070 in dem-tdb branch

2018-01-18 Thread Henning Scholland 
Hi Gerd,
thanks for merging it.

Hi Steve,
I tried it on maps, where I previously had problems with block size. Now
these maps compile well without increasing block size manually. Well done!

Henning

On 18.01.2018 19:14, Gerd Petermann wrote:
> Hi all,
>
> I've merged branch auto-block into r4070 in the dem-tdb branch.
> I've first reverted the block-size.patch and fixed two conflicts manually. 
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] r4070 in dem-tdb branch

2018-01-18 Thread Gerd Petermann 
Hi all,

I've merged branch auto-block into r4070 in the dem-tdb branch.
I've first reverted the block-size.patch and fixed two conflicts manually. 

Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Automatic block-size setting

2018-01-18 Thread Gerd Petermann 
Hi Steve,

I think best would be to ignore the --block-size option, at least for a for a 
while
like we do with --remove-short-arcs.
This makes it easier to compare results of two different binaries.

Gerd


Von: mkgmap-dev  im Auftrag von Steve 
Ratcliffe 
Gesendet: Mittwoch, 17. Januar 2018 23:41:08
An: mkgmap development
Betreff: [mkgmap-dev] Automatic block-size setting

Hi

The branch auto-block contains changes to automatically find a
suitable value for the --block-size option.

Previously the default block size was almost always OK, but with the
DEM changes the tiles are larger and a larger block size is sometimes
required.

All kinds of .img file (tiles, overview, gmapsupp and mdr) now
work the same way.

Gmapsupp.img files have always selected an appropriate block size, so
there should not be much change there.  Regular tiles may be a little
bit smaller (~1%) as a larger block size sometime results in a file
that is a bit smaller and if they are large enough to require a larger
block size that will happen automatically.

The osmmap_mdr.img files will typically be a lot smaller as
previously they had a fixed block size of 16M and wasted a lot
of space.  Now the correctly block size will be selected and the
file will be as small as possible.

Currently the --block-size option still exists, but it just determines
the smallest block size that will be used - if it is not large enough
it will be increased.  I could remove the option altogether or make it
override the automatically determined value.

..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 ; dem-dist 3342 or 3314 ?

2018-01-18 Thread Gerd Petermann 
Hi Thomas,

oops, sorry, 3342 was a typo, should be 3314  ~ 2^32 / (360 * 3600) and 9942 ~ 
2^32 / (360*1200)

Gerd


Von: mkgmap-dev  im Auftrag von Thomas 
Morgenstern 
Gesendet: Donnerstag, 18. Januar 2018 10:49:32
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r585 ; dem-dist 3342 or 3314 ?

Hi Gerd, in some posts are written different -dem-dist . Which is the right,
3314 or 3342 ? For using with --align-for-dem  1" hgt ?
thomas

--
Von: "Gerd Petermann" 
Datum: Mittwoch, 17. Januar 2018 17:42
An: 
Betreff: [mkgmap-dev] splitter r585 implements new option align-for-dem

> Hi all,
>
> see http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=585
>
> If I got that right you can use dem-dist 9942 for level 0 if hgt files are
> in 3'' resolution and splitter
> was executed with --align-for-dem=1200.
> For hgt 1'' files use dem-dist 3342 for level 0 and --align-for-dem=3600.
>
> This should reduce DEM size a lot AND increase precision.
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r585 ; dem-dist 3342 or 3314 ?

2018-01-18 Thread Thomas Morgenstern 
Hi Gerd, in some posts are written different -dem-dist . Which is the right, 
3314 or 3342 ? For using with --align-for-dem  1" hgt ?

thomas

--
Von: "Gerd Petermann" 
Datum: Mittwoch, 17. Januar 2018 17:42
An: 
Betreff: [mkgmap-dev] splitter r585 implements new option align-for-dem


Hi all,

see http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=585

If I got that right you can use dem-dist 9942 for level 0 if hgt files are 
in 3'' resolution and splitter

was executed with --align-for-dem=1200.
For hgt 1'' files use dem-dist 3342 for level 0 and --align-for-dem=3600.

This should reduce DEM size a lot AND increase precision.

Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev