Re: [mkgmap-dev] Error when running splitter with several input files, could there be keep-complete=single mode?

2021-09-01 Thread Felix Hartmann
In general I now got it fully working - the problem is if those merged
tiles are split - and not split in "pairs" but say from two tiles that
failed - which are not adjacent - 3 tiles are created (or 5) there
sometimes will be a tile spanning another tile. This works as far as I can
see in Basecamp/MapInstall/Mapsource - but gives strange impression when
selecting the tiles in MapInstall/Mapsource.
Have not yet tested it on device.

So if there is no possibility for splitter to implement a multiple single
input tile mode - I will have to resort to writing a big for loop calling
splitter separately for each crashed tile on mkgmap.jar processing. It
often can be prevented by using exactly 50% maxnodes size on resplit - but
that is not reliable of course. Basically for each tile that is 0 byte, you
need to split the original o5m, remove the mention to the input-file from
the template.args file, after each run check for a new highest mapid (or if
splitting with 50% maxnodes just add 3), always delete all 0byte files -
and after all tiles are split run mkgmap.jar again (either on the existing
.img files plus the new templates.args file (from joining all the
templates.args now newly created - also required quite a bit of logic).
It's scriptable - but really requires a lot of work. Would be great if
splitter.jar has a multiple tile input method (even more if it could
multithread that one as usual). If not the multithreading of course can be
achieved by piping each of these broken tiles into a separate process for
splitting.

On Tue, 31 Aug 2021 at 12:23, Felix Hartmann 
wrote:

> That step is very easy.
>
> You feed %FID% (if you use fid for first 4 numbers of mapid) and the new
> template.args to mkgmap.jar. No need to recreate the .img files that are
> okay. The broken ones - all 0byte ones - just delete them. If you wanna run
> mkgmap again on the same o5m input files - you join the template.args file
> from both runs - I always replace mapname with #notused so the
> template.args only contains description and the actual input file. It
> throws a warning that input files are missing - but that is allright. Of
> course you need to add in your script to delete those input files that were
> resplit.
>
> I don't know what your "app" does - but the process is pretty simple. As
> long as splitter can handle the several input files with --keep-complete
> (and just runs keep-complete on each input file but not on the totality of
> input files, plus does not try to join data from two files). And yes as
> said - if you want to run splitter on each input file separately, that
> creates loads of work - I don't know how to script that efficiently. Oh
> yeah - and your script needs to read the highest number of the maps created
> - add 1, and run mkgmap again with that number.  But yeah - scripting this
> takes some time. But it is understandable that mkgmap cannot call
> splitter.jar on it's ownThe main thing in this is getting to run
> splitter on several input files with keep-complete valid for each tile on
> it's own.
> But yeah I think osmconvert 100.o5m 11.o5m 100010.o5m
> -o=inputfile.o5m
> and then calling mkgmap with the higher mapid, and *already_created.img
> inputfile.o5m would suffice. So far I cannot see problems (as long as using
> keep-complete=true)  - you can even do a third pass. You cannot do a third
> pass if you use keep-complete=false overlap=2000
> because the overlap would create a huge mess (or better objects ending up
> twice in your map)
>
> On Tue, 31 Aug 2021 at 11:33, Eric Internet  wrote:
>
>> Also do not understand and on vacation now, so my options for
>> understanding are very limited.
>>
>> But fwiw...
>>
>> Mkgmap should read files collection from contents files. One o5m file
>> missing or 1 file extra (having different FID) is a show stopper. (Missing
>> files is not tested, assumption only).
>>
>> I create 2-6 different maps each overnight from same dataset by simply:
>> - Rename all o5m files by FID
>> - Rename Reference in two content files by filename (changed by FID).
>> This procedure takes seconds to run.
>>
>> All scripts (power management, splitter, OsmConvert, mkgmap, NSI) are
>> executed by shell.run from within a "wrapper application".
>> VB exe is: flexible, configurable, unlimited procedures and language
>> elements.
>>
>> App UI checkboxes will select which datasets to download.
>> App UI checkboxes will select which maps to be created.
>> UI settings are saved in iNI file for Unattended execution.
>> App will decide which datasets need to be merged by OsmConvert.
>> App will check preconditions.
>> App will run preprocess procedures (f.i. Rename by FID).
>> App will wait for recent "latest" dataset.
>> App will retry a failed download for n-times.
>> App will decide what to do if a dataset is missing or outdated.
>> App will verify each step to be successful (or not).
>> App will install correctly created maps (read log files) into BaseCamp.
>> Installer exe 

Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null

2021-09-01 Thread Felix Hartmann
Okay - this error can be solved by removing the "Input that do not exist
from the template.args input file. - e.g. if 85610002.o5m does not exist
need to remove the line
input-file: 85610002.o5m
from the template.args file.

On Tue, 31 Aug 2021 at 21:11, Felix Hartmann 
wrote:

> Partly related to the other problem - if I run mkgmap on the splitted
> files - I get this error. All .img files seem to be fine, but the
> gmpsupp.img is created with 0 bit size.
>
> This is not happening every time I resplit tiles. In this case two tiles
> were joined by osmconvert - and splitter splits it into three tiles. Then
> all .img files are created correctly, but the mapset.tdb / mapset.img files
> are not created - and the gmapsupp.img has 0 size, the gmapi files is also
> not created.
>
> I guess this happens because splitter somehow made a melange of the two
> input files into three?
>
> I really think the only thing missing in resplitting the too big tiles
> from mkgmap is an appropriate keep-complete mode - that keeps each input
> tile complete - but does not join input tiles. So osmconvert merging the
> input tiles is a bad idea even though it mostly works.
>
> C:\openmtbmap\maps>start /belownormal /b /wait java -jar
> -XX:+AggressiveHeap -XX:StringTableSize=103 -Xms5000M -Xmx43000M
> C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area
> --code-page=1251 "--style-file=C:\openmtbmap\buildings_style"
>  --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds
> --ignore-turn-restrictions --merge-lines --allow-reverse-merge
> --transparent --draw-priority=28 --add-pois-to-areas
> --simplify-polygons=23:4,22:7,21:8
> --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt"
> --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt"
> --add-boundary-nodes-at-admin-boundaries=2
> --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values
> --polygon-size-limits="24:16, 23:14, 22:12, 21:11"
> --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru
> --country-name=russia --mapname=8580 --family-id=8580 --product-id=1
> --series-name="buildings_russia_31.08.2021_NU_Local"
> --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi
> --gmapsupp --overview-mapname=mapsetb --keep-going
> --area-name="russia_31.08.2021_NU_Local_buildings" -c
> D:\openmtbmap\maps\template.russiab buildru.typ
> [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size,
> will use initial heap size.
> Mkgmap version 4806M
> Time started: Tue Aug 31 21:00:52 CEST 2021
> SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file
> 'D:\openmtbmap\maps\85800012.o5m' doesn't exist
> SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file
> 'D:\openmtbmap\maps\85800024.o5m' doesn't exist
> Number of MapFailedExceptions: 0
> Exception in thread "main" java.lang.NullPointerException: Cannot invoke
> "String.endsWith(String)" because the return value of
> "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
> at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620)
> at
> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
> at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146)
> at uk.me.parabola.mkgmap.main.Main.main(Main.java:117)
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] How to create an empty overview map (for transparent maps)

2021-09-01 Thread Felix Hartmann
If you create a separate map for buildings or contourlines only - usually
there is no need for an overview map - or better no need for any content in
the overview map - or am I missing something?

Because those maps sometimes may have very large tiles, but very few levels
(maybe only 24) - mkgmap creates an overview map at the next resolution -
and will then often throw warnings such as:

SEVERE (OverviewBuilder): Tile selection (0x4a) polygon for tile 8490
cannot be written in level 0 resolution 21, using 20 instead


There is still a need for an empty overview map - as without it you cannot
show the map in Basecamp/Mapsource - or transfer it with MapInstall.
I know the overview map is only empty 0x4a polygons - just wondered if
there is a better solution...

--overview-levels=
is not given.

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev