Re: [mkgmap-dev] mkgmap ToDo list
Hi all, I've coded a quick hack which seems to improve the ratios. On the other hand, I don't see these large differences between smallest and largest img file. What part of the world should I try? Do you use the precomp-sea parameter in splitter? Gerd Date: Wed, 30 Apr 2014 14:59:32 +0200 From: extremecar...@gmail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] mkgmap ToDo list if that doesn't seriously (more than 30-40%) slow down the splitter, I assume it would be much better... On 30.04.2014 14:06, Gerd Petermann wrote: Hi, if I got that right the number of nodes is not highly correlated to the img size, so the max-nodes value is not a good estimate. I assume the reason is that nodes which belong to roads produce a lot more bytes in the img file compared to nodes which are parts of shapes or other non-routable ways, not talking about nodes which are simply ignored by the style. So, a possible solution in splitter could be to parse the ways before reading the nodes and save all nodeids which belong to ways with highway=*. If these nodes are refered by more than one way with highway=* we assume that they will be routing nodes. These special nodes could be counted e.g. 10 times to give a better estimate. Gerd > Date: Wed, 30 Apr 2014 13:36:19 +0200 > From: o...@na1400.info > To: mkgmap-dev@lists.mkgmap.org.uk > Subject: Re: [mkgmap-dev] mkgmap ToDo list > > Multithreading the tile rendering for a single map is indeed a bit > difficult and I gave it up because you need to keep track which image > id's are already in use. Since I provide multiple maps the work-around > is running a few scripts parallel, which is also a crude form of > multithreading. > > The script language is PHP and it doesn't run on Windows without some > changes ('/' vs '\' in paths, 'rm -rf', that sort of stuff). Never tried it. > > To get a better optimum in file size, using the process I described > earlier, you could start off with a huge --max-nodes setting and then > 'search' for the highest --max-nodes that works for each specific area. > > On 30/04/2014 11:49, Felix Hartmann wrote: > > I would love if there was a possibility that you pass the used max-nodes > > value to mkgmap. > > When mkgmap is compiling the maps, then after the .img is created it > > should check > > a) did it crash due to too many max-nodes > > b) for me not important - but for others with very old GPS, etrex 10, > > ---> is tile bigger than X (usually 8) MB. > > > > if a) or b) true, then pass the file back to splitter and split with 60% > > of maxnodes - and compile the resulting .img files again. Should it fail > > again, use 40%, again 25%... Sometimes there are awful tiles, that need > > supersmall max-nodes till they compile, however lately (last 1-2 years) > > I never encountered them anymore. I think that happened rather due to a > > but in splitter/mgkmap that is fixed now. > > > > okay, you could also do this with a script, but it gets rather > > complicated to multithread it (you need to wait till mgkmap finished > > compiling all .img files - and run mkgmap first without address index to > > save time) and do some clever routines on making sure that the map id > > (e.g. 6340.img) stay correct. Even more complicated to have > > consequent map id... > > > > For europe with a fixed max-node I get tiles from 1.9MB up to 18MB. > > That's factor 9 - so it's huge... > > If I could narrows that down easily to 8-18MB - without getting tiles > > crashing due to too high max-nodes values, that would be sweet. > > > > > > > > As for the scripts - would they run on windows too? - What programming > > language are they in? > > > > > > On 29.
Re: [mkgmap-dev] r3251 in performance2 branch
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am 02.05.2014 23:21, schrieb Gerd Petermann: > Hi all, > > please test / review the changes in the branch, because I don't > plan any further optimizations for now. You should see no different > output compared with r3248 in trunk, but around 15% faster > execution time and also a reduction in peak memory usage. For me it seems to be a great improvement. Henning -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) iQEcBAEBAgAGBQJTZJ4nAAoJEKXggIeC16WPzp8H/A2wrljfP2wamkT0bAvTLq3A b4CWMJP5kQHNukgP6KNCtiEIf9/aDnlKJ36Z2c14NNnIQI6/9PMwVcqSn1C2gfh3 NAd/qhbc+3Syr8P9mqVLoP1OFT8OCqTGjSqbBhdN3Qaqa+bDuD6Mj2Th1H85w0pz kEGjIagjNlrRRuvLkkjXJiRykc9Rxtp5mMB2aSNvGH9g/nb8Dhi9E/airp+pji6+ SwqqMuOe88FnYFiIgiidp0QSvWNwI4lt05572wmIvE9sDGRVfe9zYEzAM9mMRnFn bNSl0ZjVbZuPIz0gBzdGq7wwh7VIHzma5JBIr4tFNMRs3y+6Lyvu7ecJBCBiuwg= =iSDS -END PGP SIGNATURE- ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Selon Stéphane MARTIN : > Hi, Hi Stéphane, hi all I reply to Stéphane but I think everyone should read and may reply as I have general questions. > Does this proposal makes sense for France ? > According to > http://wiki.openstreetmap.org/wiki/OSM_tags_for_routing/Access-Restrictions#France I don't understand this page as you do. I read this table as : which traffic mode is allowed *by default* (when the OSM way has no access tag defined, only the highway tag). > > # France (FRA) > > highway=trunk & mkgmap:country=FRA > { add bicycle=no; add foot=no } Correct, we can duplicate this one for motorway also. > highway=cycleway & bicycle=designated & mkgmap:country=FRA > { add foot=no } > highway=cycleway & mkgmap:country=FRA > { set mkgmap:foot=yes; } As I explained in the beginning, the rules are : highway=cycleway & mkgmap:country=FRA { set mkgmap:access=no; set mkgmap:bicycle=yes;} highway=cycleway & foot=yes & mkgmap:country=FRA { set mkgmap:access=no; set mkgmap:bicycle=yes; set mkgmap:foot=yes; } I don't even think the second rule is needed. > highway=bridleway & horse=designated & mkgmap:country=FRA > { add bicycle=no; add foot=no } > highway=bridleway & mkgmap:country=FRA > { set mkgmap:foot=yes; set mkgmap:bicycle=yes; } This one is tricky, I'm no horse rider but I think bridleways have no legal definition in France. I understand there are only paths which may be allowed to horse riders. I'd consider them as highway=path. But this should be discussed with the French OSM community, not here. So for now, let's apply the wiki table : highway=bridleway & mkgmap:country=FRA { set mkgmap:access=no; set mkgmap:horse=yes;} BTW, I don't remember mkgmap:horse exists, correct ? So what should we do with them ? Do we need to define rules for all the other highway tag values (primary, secondary, tertiary, unclassified, residential, living_street, track, footway and pedestrian) ? Are they handled by mkgmap by default ? > > Steph Paco ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Hi Paco, Please check: Why do you use the tags mkgmap:horse =* and mkgmap:access=* ? They are not used in mkgmap. The img format doesn't know horses. Gerd > Date: Sat, 3 May 2014 10:00:05 +0200 > From: paco.ty...@free.fr > To: mkgmap-dev@lists.mkgmap.org.uk > Subject: Re: [mkgmap-dev] access_country for default style > > Selon Stéphane MARTIN : > > > Hi, > Hi Stéphane, hi all > > I reply to Stéphane but I think everyone should read and may reply as I have > general questions. > > > Does this proposal makes sense for France ? > > According to > > > http://wiki.openstreetmap.org/wiki/OSM_tags_for_routing/Access-Restrictions#France > > I don't understand this page as you do. I read this table as : which traffic > mode is allowed *by default* (when the OSM way has no access tag defined, only > the highway tag). > > > > > # France (FRA) > > > > highway=trunk & mkgmap:country=FRA > > { add bicycle=no; add foot=no } > > Correct, we can duplicate this one for motorway also. > > > highway=cycleway & bicycle=designated & mkgmap:country=FRA > > { add foot=no } > > highway=cycleway & mkgmap:country=FRA > > { set mkgmap:foot=yes; } > > As I explained in the beginning, the rules are : > highway=cycleway & mkgmap:country=FRA > { set mkgmap:access=no; set mkgmap:bicycle=yes;} > highway=cycleway & foot=yes & mkgmap:country=FRA >{ set mkgmap:access=no; set mkgmap:bicycle=yes; set mkgmap:foot=yes; } > > I don't even think the second rule is needed. > > > highway=bridleway & horse=designated & mkgmap:country=FRA > > { add bicycle=no; add foot=no } > > highway=bridleway & mkgmap:country=FRA > > { set mkgmap:foot=yes; set mkgmap:bicycle=yes; } > > This one is tricky, I'm no horse rider but I think bridleways have no legal > definition in France. I understand there are only paths which may be allowed > to > horse riders. I'd consider them as highway=path. But this should be discussed > with the French OSM community, not here. So for now, let's apply the wiki > table > : > > highway=bridleway & mkgmap:country=FRA >{ set mkgmap:access=no; set mkgmap:horse=yes;} > > BTW, I don't remember mkgmap:horse exists, correct ? So what should we do with > them ? > > Do we need to define rules for all the other highway tag values (primary, > secondary, tertiary, unclassified, residential, living_street, track, footway > and pedestrian) ? Are they handled by mkgmap by default ? > > > > > > Steph > > Paco > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Paco wrote > > # France (FRA) > > > > highway=trunk & mkgmap:country=FRA > > { add bicycle=no; add foot=no } > > Correct, we can duplicate this one for motorway also. Motorways already get { add bicycle=no; add foot=no } so for motorways it is not needed to add another rule for France. Same for all other highways, only if they are different than the mkgmap style, we need to add those rules. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
On 03.05.2014 10:16, Gerd Petermann wrote: Hi Paco, Please check: Why do you use the tags mkgmap:horse =* and mkgmap:access=* ? They are not used in mkgmap. The img format doesn't know horses. Gerd well - at least mkgmap:access would be really useful... (and not as a shortcut...). I was going to write a lengthy description why finalize cannot replace it... At least my style is still pretty badly corrupting the maps, because it's not possible with any trick to replace it's old functionality... (problems are related to continue and continue with_actions). In the end I would end up with a finalize section some hundred lines long, just in order to try to restore the old behaviour. compat_lines is only working fully for default style... I will explain that in some days if I find time, cause it's quite complicated... - the main problem is that there is no intermediate finalize. Actually to fully replace it's old behaviour I would need a complex structure of several finalize blocks. One single finalize block at the end doesn't help me at all. Is it possible to have several finalize sections - and each subsequent only finalizes the lines in between the last finalize? Furthermore, I still think mkgmap:access is the most important key, I'm not so sure which mkgmap:bicycle,foot, key Basecamp v4 actually interprets and in which profile. Needs some testing now that is supposedly works... mgmap:horse of course makes no sense.. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] r3251 in performance2 branch
@WanMil, Steve: Please let me know if you see problems with the usage of the new TagDict class or the RuleSet.compile() method. The latter uses the toString() method to identify what an Op is doing. Is that okay for you or should I create a different method like getOpIdentifier() to make it clearer that the method should not be changed without good reason? Hi Gerd, I have no time to have a look at the changes. It sounds like a good idea to use the getOpIdentifier() instead of toString(). WanMil ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
On 03.05.2014 10:16, Gerd Petermann wrote: Hi Paco, Please check: Why do you use the tags mkgmap:horse =* and mkgmap:access=* ? They are not used in mkgmap. The img format doesn't know horses. Gerd well - at least mkgmap:access would be really useful... (and not as a shortcut...). I was going to write a lengthy description why finalize cannot replace it... At least my style is still pretty badly corrupting the maps, because it's not possible with any trick to replace it's old functionality... (problems are related to continue and continue with_actions). In the end I would end up with a finalize section some hundred lines long, just in order to try to restore the old behaviour. compat_lines is only working fully for default style... I will explain that in some days if I find time, cause it's quite complicated... - the main problem is that there is no intermediate finalize. Actually to fully replace it's old behaviour I would need a complex structure of several finalize blocks. One single finalize block at the end doesn't help me at all. Is it possible to have several finalize sections - and each subsequent only finalizes the lines in between the last finalize? Furthermore, I still think mkgmap:access is the most important key, I'm not so sure which mkgmap:bicycle,foot, key Basecamp v4 actually interprets and in which profile. Needs some testing now that is supposedly works... mgmap:horse of course makes no sense.. Felix, if you need mkgmap:access you can add it yourself and add the rule mkgmap:access=* { addaccess ${mkgmap:access} } to your finalize section. If that doesn't help please post a short and understandable example why it is required. WanMil ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Hi Paco, It seems that there is no consensus for the French OSM community: http://gis.19327.n5.nabble.com/Acces-de-certaines-voies-en-France-td5804000.html Furthermore, I'm realizing that "designated" can be associated with an other "designated" or with an other tag important for routing. Therefore, permissive rules would be better here and some restrictions can be added, e.g. in "lines", depending the wanted routing. # France (FRA) highway=trunk & mkgmap:country=FRA { add bicycle=no; add foot=no } highway=cycleway & mkgmap:country=FRA { add foot=yes } highway=bridleway & mkgmap:country=FRA { add bicycle=yes; add foot=yes } Do you agree with that ? Steph Le 03/05/2014 05:00, paco.ty...@free.fr a écrit : > Selon Stéphane MARTIN : > >> Hi, > Hi Stéphane, hi all > > I reply to Stéphane but I think everyone should read and may reply as I have > general questions. > >> Does this proposal makes sense for France ? >> According to >> > http://wiki.openstreetmap.org/wiki/OSM_tags_for_routing/Access-Restrictions#France > > I don't understand this page as you do. I read this table as : which traffic > mode is allowed *by default* (when the OSM way has no access tag defined, only > the highway tag). > >> >> # France (FRA) >> >> highway=trunk & mkgmap:country=FRA >> { add bicycle=no; add foot=no } > > Correct, we can duplicate this one for motorway also. > >> highway=cycleway & bicycle=designated & mkgmap:country=FRA >> { add foot=no } >> highway=cycleway & mkgmap:country=FRA >> { set mkgmap:foot=yes; } > > As I explained in the beginning, the rules are : > highway=cycleway & mkgmap:country=FRA > { set mkgmap:access=no; set mkgmap:bicycle=yes;} > highway=cycleway & foot=yes & mkgmap:country=FRA >{ set mkgmap:access=no; set mkgmap:bicycle=yes; set mkgmap:foot=yes; } > > I don't even think the second rule is needed. > >> highway=bridleway & horse=designated & mkgmap:country=FRA >> { add bicycle=no; add foot=no } >> highway=bridleway & mkgmap:country=FRA >> { set mkgmap:foot=yes; set mkgmap:bicycle=yes; } > > This one is tricky, I'm no horse rider but I think bridleways have no legal > definition in France. I understand there are only paths which may be allowed > to > horse riders. I'd consider them as highway=path. But this should be discussed > with the French OSM community, not here. So for now, let's apply the wiki > table > : > > highway=bridleway & mkgmap:country=FRA >{ set mkgmap:access=no; set mkgmap:horse=yes;} > > BTW, I don't remember mkgmap:horse exists, correct ? So what should we do with > them ? > > Do we need to define rules for all the other highway tag values (primary, > secondary, tertiary, unclassified, residential, living_street, track, footway > and pedestrian) ? Are they handled by mkgmap by default ? ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] The --route option
Gerd Petermann writes: > I think we should remove (ignore) the --route option, as it is mandantory. > If possible, we should allow to create maps that don't contain > a NOD section, I think this is wanted by some users. I'm not sure what you mean. It sounds like having basic/advanced/routing is desireable from the list consensus. But I would suggest that routable maps be the default, when mkgmap is invoked without special arguments. Perhaps keep --net and --route, but allow --no-net and --no-route and make --net and --route default (and of course make route imply net) pgpdKYO7YjpWV.pgp Description: PGP signature ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] The --route option
Hi Greg, > > I think we should remove (ignore) the --route option, as it is mandantory. > > If possible, we should allow to create maps that don't contain > > a NOD section, I think this is wanted by some users. > > I'm not sure what you mean. It sounds like having > basic/advanced/routing is desireable from the list consensus. But I > would suggest that routable maps be the default, when mkgmap is invoked > without special arguments. Perhaps keep --net and --route, but allow > --no-net and --no-route and make --net and --route default (and of > course make route imply net) What I meant is this: The current implementation of mkgmap does more or less assume that --route is used. Many functions are not executed without that option, although they are not needed for routing, only for the creation of the NET data. For example, --housenumber has no effect without --route, but if I got that right you may want to be able to search for housenumbers without being able to route to them. NET data is not written without --route. In the current implementation it is not simple to write NET data without also writing NOD data. I agree that we should support NET without NOD. Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] r3251 in performance2 branch
Hi WanMil, > > @WanMil, Steve: Please let me know if you see problems with > > the usage of the new TagDict class or the RuleSet.compile() > > method. The latter uses the toString() method to identify what > > an Op is doing. Is that okay for you or should I > > create a different method like getOpIdentifier() to > > make it clearer that the method should not be changed without > > good reason? > > I have no time to have a look at the changes. > It sounds like a good idea to use the getOpIdentifier() instead of > toString(). I thought about this again. A better name for the method would be getExpression(), but I think toString() is already used like that, so the new method simply means duplication of code. So, I have only replaced a few System.err.println(...) by throw new ExitException(...) with r3252. Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Selon Stéphane MARTIN : > Hi Paco, Hi Stéphane, > It seems that there is no consensus for the French OSM community: > http://gis.19327.n5.nabble.com/Acces-de-certaines-voies-en-France-td5804000.html I read this thread, I think there's too much misunderstanding and offtopic issues to get something meaningful out of it. > Furthermore, I'm realizing that "designated" can be associated with an > other "designated" or with an other tag important for routing. I think you nailed it, much of the misunderstanding and our disagreement comes from this tag. I understand the key designated as : this way has been made specifically for this vehicle, it should be used in priority before the surrounding ones. What do you think designated is used for ? Following this definition, a way tagged with (highway=bridleway & horse=designated) has redundancy because a bridleway is, by definition, made specifically for horses. > > Therefore, permissive rules would be better here and some restrictions > can be added, e.g. in "lines", depending the wanted routing. > > # France (FRA) > > highway=trunk & mkgmap:country=FRA { add bicycle=no; add foot=no } Agree > highway=cycleway & mkgmap:country=FRA { add foot=yes } Disagree. In French law, a cycleway is not allowed to pedestrians unless signed otherwise. highway=cycleway & mkgmap:country=FRA { add foot=no } (if this rule is not already applied by default in mkgmap, I don't know the default ones, are they posted somewhere ?) > highway=bridleway & mkgmap:country=FRA { add bicycle=yes; add foot=yes } Agree Of course, these rules are to be applied at the beginning of the mkgmap processing so a superseding attribute (for example foot=yes for a cycleway) can be applied correctly. > > Do you agree with that ? > > Steph > > Le 03/05/2014 05:00, paco.ty...@free.fr a écrit : > > Selon Stéphane MARTIN : > > > >> Hi, > > Hi Stéphane, hi all > > > > I reply to Stéphane but I think everyone should read and may reply as I > have > > general questions. > > > >> Does this proposal makes sense for France ? > >> According to > >> > > > http://wiki.openstreetmap.org/wiki/OSM_tags_for_routing/Access-Restrictions#France > > > > I don't understand this page as you do. I read this table as : which > traffic > > mode is allowed *by default* (when the OSM way has no access tag defined, > only > > the highway tag). > > > >> > >> # France (FRA) > >> > >> highway=trunk & mkgmap:country=FRA > >> { add bicycle=no; add foot=no } > > > > Correct, we can duplicate this one for motorway also. > > > >> highway=cycleway & bicycle=designated & mkgmap:country=FRA > >> { add foot=no } > >> highway=cycleway & mkgmap:country=FRA > >> { set mkgmap:foot=yes; } > > > > As I explained in the beginning, the rules are : > > highway=cycleway & mkgmap:country=FRA > > { set mkgmap:access=no; set mkgmap:bicycle=yes;} > > highway=cycleway & foot=yes & mkgmap:country=FRA > >{ set mkgmap:access=no; set mkgmap:bicycle=yes; set mkgmap:foot=yes; } > > > > I don't even think the second rule is needed. > > > >> highway=bridleway & horse=designated & mkgmap:country=FRA > >> { add bicycle=no; add foot=no } > >> highway=bridleway & mkgmap:country=FRA > >> { set mkgmap:foot=yes; set mkgmap:bicycle=yes; } > > > > This one is tricky, I'm no horse rider but I think bridleways have no legal > > definition in France. I understand there are only paths which may be > allowed to > > horse riders. I'd consider them as highway=path. But this should be > discussed > > with the French OSM community, not here. So for now, let's apply the wiki > table > > : > > > > highway=bridleway & mkgmap:country=FRA > >{ set mkgmap:access=no; set mkgmap:horse=yes;} > > > > BTW, I don't remember mkgmap:horse exists, correct ? So what should we do > with > > them ? > > > > Do we need to define rules for all the other highway tag values (primary, > > secondary, tertiary, unclassified, residential, living_street, track, > footway > > and pedestrian) ? Are they handled by mkgmap by default ? > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] access_country for default style
Hi Paco, paco.tyson wrote >> highway=cycleway & mkgmap:country=FRA { add foot=yes } > highway=cycleway & mkgmap:country=FRA { add foot=no } > (if this rule is not already applied by default in mkgmap, I don't know > the > default ones, are they posted somewhere ?) The default style is part of the mkgmap package, you can find it in mkgmap-r\examples\styles\default But it is only used if you don't specify another one with --style-file. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/access-country-for-default-style-tp5804979p5805031.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev