[josm-dev] JOSM download with tile numbers?
Hi, one of the tabs in JOSMs download can be used to specify a download area with "tile numbers". Is there anybody actually using this? Who would actually know tile numbers? Regards Karl ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] JOSM download with tile numbers?
On Sat, 31 Oct 2009 09:57:12 +0200, Karl Guggisberg wrote: > Hi, > > one of the tabs in JOSMs download can be used to specify a download area > with "tile numbers". Is there anybody actually using this? Who would > actually know tile numbers? > The rendered map is stored as tiles, and you can see the tile number from the filenames. The directory structure is /zoom/x/y.png I haven't used the feature, but one case I think this might be useful, is when debugging some t...@h rendering problem (although t...@h downloads a bit more than exactly a tile). eg. a way that has a long segment that doesn't show up in certain tile. Regards Teemu Koskinen ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
[josm-dev] Tutorial/docs for plugin-programmers?
I have a few ideas for plugins that I'd like to try out, but I need some hand-holding getting started. I've been looking around for a tutorial or write-up on how to do it but I haven't found any. A "skeleton" plugin would also be useful. I am quite experienced in C++ and Python, and understand OO programming, but I've never worked seriously with Java. Cheers, Morten ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Tutorial/docs for plugin-programmers?
> I have a few ideas for plugins that I'd like to try out, but I need > some hand-holding getting started. I've been looking around for a > tutorial or write-up on how to do it but I haven't found any. A > "skeleton" plugin would also be useful. The best place to start is the introductory page at http://josm.openstreetmap.de/wiki/DevelopingPlugins . There is an example template plugin that helps. Just ask here for anything that's not covered. The general JOSM developer's guide has been recently updated and there are 2 helpful videos. http://josm.openstreetmap.de/wiki/DevelopersGuide (I find that Eclipse is great when it works, but it hides much of the activity which makes it hard to troubleshoot when things go wrong) ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
[josm-dev] QuadBuckets in Dataset
Hi, QuadBuckets is used for some time as storage for nodes and ways in Dataset but I think it doesn't bring any benefit for JOSM in current state. Ability to do fast search for bounding box will be usefull in future, but right now it can't be used because it's not sure that QuadBuckets will be reindexed after change of node coordinates. In any case, I don't think QuadBuckets or any other concreate storage class should be exposed as public field in Dataset. There should be only unmodifiable collection, all changes should go through addPrimitive/removePrimitive and possibly other methods. So the first suggestion is replace QuadBuckets with Collection again. Another thing is what class should be used internally as a storage for nodes and ways. The most common operation is simply iterarating over all elements - that's the task where QuadBuckects is more than three times slower than ArrayList. and two times slower than Storage class. Another operation is looking for element based on id. That's currently used in methods like Way.load(), but it will be also useful in other places, like in currently hot selection code. I think selection should be id based - when Dataset.setSelected(Node) is called then node that exist in the Dataset and has the same id as argument should be selected, not neccessary the same instance of Node passed as an argument. That will preserve backwards compatibility with old code, that relies on the fact that OsmPrimitive.cloneFrom preserved selection flag and it will be also useful for replacing OsmPrimitive with PrimitiveData classes. To summarize, I would like to do following: 1) Replace public QuadBuckets nodes in Dataset with Collection getNodes(), that will return Collections.unmodifiableCollection of nodes 2) Replace for now QuadBuckets in Dataset with class more suitable for iterating and looking for ids - Storage* class from josm-ng looks like a good candidate. QuadBuckets can be used again when it makes sense (maybe both Storage and QuadBuckets can be used at the same time, as each class is useful for differrent type of operations) Are there any complains? * Storage class is basically memory effective hash map implementation. -- Jiri ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
On Sat, Oct 31, 2009 at 3:28 PM, Jiri Klement wrote: > Are there any complains? I'd just like to note that as a JOSM user I appreciated QuadBuckets since I could run the Validator on large datasets for the first time without JOSM taking forever (& having to be killed). No matter what code ends up being the primary storage class for JOSM I'd appreciate the Validator having access to something like QuadBuckets. ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
Hi, I'd also like to see a kind of spatial index but I agree with Jiri that QuadBucket is currently only used in validator. And I fully agree with the steps Jiri proposes. Even if QuadBucket was disabled for a short time we would gain a lot with Jiris moves. The major goal is still to turn DataSet in a properly managed and consistent object storage which allows for efficient traversal from childs to parents. -- Karl -Ursprüngliche Nachricht- Von: josm-dev-boun...@openstreetmap.org [mailto:josm-dev-boun...@openstreetmap.org] Im Auftrag von Jiri Klement Gesendet: Samstag, 31. Oktober 2009 16:28 An: josm-dev Betreff: [josm-dev] QuadBuckets in Dataset Hi, QuadBuckets is used for some time as storage for nodes and ways in Dataset but I think it doesn't bring any benefit for JOSM in current state. Ability to do fast search for bounding box will be usefull in future, but right now it can't be used because it's not sure that QuadBuckets will be reindexed after change of node coordinates. In any case, I don't think QuadBuckets or any other concreate storage class should be exposed as public field in Dataset. There should be only unmodifiable collection, all changes should go through addPrimitive/removePrimitive and possibly other methods. So the first suggestion is replace QuadBuckets with Collection again. Another thing is what class should be used internally as a storage for nodes and ways. The most common operation is simply iterarating over all elements - that's the task where QuadBuckects is more than three times slower than ArrayList. and two times slower than Storage class. Another operation is looking for element based on id. That's currently used in methods like Way.load(), but it will be also useful in other places, like in currently hot selection code. I think selection should be id based - when Dataset.setSelected(Node) is called then node that exist in the Dataset and has the same id as argument should be selected, not neccessary the same instance of Node passed as an argument. That will preserve backwards compatibility with old code, that relies on the fact that OsmPrimitive.cloneFrom preserved selection flag and it will be also useful for replacing OsmPrimitive with PrimitiveData classes. To summarize, I would like to do following: 1) Replace public QuadBuckets nodes in Dataset with Collection getNodes(), that will return Collections.unmodifiableCollection of nodes 2) Replace for now QuadBuckets in Dataset with class more suitable for iterating and looking for ids - Storage* class from josm-ng looks like a good candidate. QuadBuckets can be used again when it makes sense (maybe both Storage and QuadBuckets can be used at the same time, as each class is useful for differrent type of operations) Are there any complains? * Storage class is basically memory effective hash map implementation. -- Jiri ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
> I'd just like to note that as a JOSM user I appreciated QuadBuckets > since I could run the Validator on large datasets for the first time > without JOSM taking forever (& having to be killed). I've noticed that validator plugin is using QuadBuckets, but in my opinion UnconnectedWays test will miss some cases in case they're caused by moved nodes, because MoveCommand doesn't reindex QuadBuckets. Or am I missing something? ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
On Sat, 2009-10-31 at 16:28 +0100, Jiri Klement wrote: > To summarize, I would like to do following: > 1) Replace public QuadBuckets nodes in Dataset with > Collection getNodes(), that will return > Collections.unmodifiableCollection of nodes That's fine with me, as long as you also add a few search functions to the DataSet to replace the ones that are already in use. I think that's the validator and the selection code. > 2) Replace for now QuadBuckets in Dataset with class more suitable for > iterating and looking for ids - Storage* class from josm-ng looks like > a good candidate. QuadBuckets can be used again when it makes sense > (maybe both Storage and QuadBuckets can be used at the same time, as > each class is useful for differrent type of operations) > > Are there any complains? I'm biased, but I'd really like to keep it, please. I'm using it in at least two places I mentioned above. I just committed code to the validator's UnconnectedWays test that takes a test from 3 minutes on a large data set down to less than 2 seconds. It took processing mouse clicks on the same data set from taking over a second to being instantaneous. There are certainly operations for which it is not optimal. But, as its search capabilities grow more widely, we will have less and less need to iterate over the data set. We can also tune things like the bucket size to ensure that iterations get cheaper. If you want a cheaper-to-iterate data structure, I have a suggestion. Instead of actually storing OsmPrimitives in QuadBuckets, we have an ArrayList or TreeSet sorted on id. We store the id in the QuadBuckets leaves. The lookup by id is actually a pretty rare operation in my experience. I haven't seen it show up in any performance profiles, and I've done a tone of those. -- Dave ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
On Sat, 2009-10-31 at 17:08 +0100, Jiri Klement wrote:
> > I'd just like to note that as a JOSM user I appreciated QuadBuckets
> > since I could run the Validator on large datasets for the first time
> > without JOSM taking forever (& having to be killed).
>
> I've noticed that validator plugin is using QuadBuckets, but in my
> opinion UnconnectedWays test will miss some cases in case they're
> caused by moved nodes, because MoveCommand doesn't reindex
> QuadBuckets. Or am I missing something?
One suggestion was to use the UndoRedoHandler code to catch
modifications and reindex the primitives. How about something like
this?
Index: src/org/openstreetmap/josm/data/UndoRedoHandler.java
===
--- src/org/openstreetmap/josm/data/UndoRedoHandler.java(revision 2355)
+++ src/org/openstreetmap/josm/data/UndoRedoHandler.java(working copy)
@@ -3,11 +3,13 @@
import java.util.Iterator;
import java.util.LinkedList;
+import java.util.HashSet;
import java.util.Stack;
import org.openstreetmap.josm.Main;
import org.openstreetmap.josm.command.Command;
import org.openstreetmap.josm.data.osm.DataSet;
+import org.openstreetmap.josm.data.osm.OsmPrimitive;
import org.openstreetmap.josm.gui.layer.Layer;
import org.openstreetmap.josm.gui.layer.OsmDataLayer;
import org.openstreetmap.josm.gui.layer.Layer.LayerChangeListener;
@@ -38,16 +40,32 @@
public void addNoRedraw(final Command c) {
c.executeCommand();
commands.add(c);
+recordModified(c);
redoCommands.clear();
}
-public void afterAdd() {
+HashSet modifiedPrimitives = new HashSet();
+void recordModified(Command c)
+{
+LinkedList deleted = new LinkedList();
+LinkedList added = new LinkedList();
+LinkedList modified = new LinkedList();
+c.fillModifiedData(modified, deleted, added);
+modifiedPrimitives.addAll(modified);
+}
+public void sendNotifications()
+{
if (Main.map != null && Main.map.mapView.getActiveLayer() instanceof
OsmDataLayer) {
OsmDataLayer data =
(OsmDataLayer)Main.map.mapView.getActiveLayer();
data.fireDataChange();
}
+
Main.main.getCurrentDataSet().notifyPrimitiveChange(modifiedPrimitives);
+modifiedPrimitives.clear();
fireCommandsChanged();
+}
+public void afterAdd() {
+sendNotifications();
// the command may have changed the selection so tell the listeners
about the current situation
Main.main.getCurrentDataSet().fireSelectionChanged();
}
@@ -69,11 +87,8 @@
final Command c = commands.removeLast();
c.undoCommand();
redoCommands.push(c);
-if (Main.map != null && Main.map.mapView.getActiveLayer() instanceof
OsmDataLayer) {
-OsmDataLayer data =
(OsmDataLayer)Main.map.mapView.getActiveLayer();
-data.fireDataChange();
-}
-fireCommandsChanged();
+recordModified(c);
+sendNotifications();
Main.main.getCurrentDataSet().setSelected();
}
@@ -86,12 +101,9 @@
return;
final Command c = redoCommands.pop();
c.executeCommand();
+recordModified(c);
commands.add(c);
-if (Main.map != null && Main.map.mapView.getActiveLayer() instanceof
OsmDataLayer) {
-OsmDataLayer data =
(OsmDataLayer)Main.map.mapView.getActiveLayer();
-data.fireDataChange();
-}
-fireCommandsChanged();
+sendNotifications();
}
public void fireCommandsChanged() {
Index: src/org/openstreetmap/josm/data/osm/DataSet.java
===
--- src/org/openstreetmap/josm/data/osm/DataSet.java(revision 2355)
+++ src/org/openstreetmap/josm/data/osm/DataSet.java(working copy)
@@ -645,4 +645,62 @@
}
return ret;
}
+
+List waysUsingNode(Node n)
+{
+List possible_ways = ways.search(n.getCoor(), 0.0);
+List result = new ArrayList();
+for (Way w : possible_ways) {
+if (!w.containsNode(n))
+continue;
+result.add(w);
+}
+return result;
+}
+
+void reIndex(Way w)
+{
+ways.remove(w);
+ways.add(w);
+}
+
+void reIndex(Node n)
+{
+nodes.remove(n);
+nodes.add(n);
+}
+
+void reIndexPrimitives(Collection modified)
+{
+for (OsmPrimitive o : modified) {
+if (o instanceof Way)
+reIndex((Way)o);
+if (o instanceof Node)
+reIndex((Node)o);
+}
+}
+
+/*
+ * When nodes and ways get modified, their spatial properties may
+ * change. This means that their locations in the spatially-indexed
+ * QuadBuckets structures may change. When this happens, delete
+ * and reinsert them.
+ */
+
Re: [josm-dev] QuadBuckets in Dataset
> One suggestion was to use the UndoRedoHandler code to catch > modifications and reindex the primitives. How about something like > this? I'm not sure if the code will work for one moved node. You first call reIndexPrimitives(modified) that might move node to different bucket and then try to get ways, that still think the node is in the old buckets. Anyway after some modifications it should work, at least as a temporary solution. So I will replace direct access to QuadBuckets with getNodes() method returning read-only collection and add search method. Internally Dataset will use both QuadBuckets for fast access by coordinates and Storage for fast access using id and iterations. Because dataset modifications will be possible only through Dataset methods, it shouldn't be problem to keep the two lists synchronized. -- Jiri ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
[josm-dev] JOSM Stable
Hello, again there is a lot of time since last release and a new tested should come. Currently I consider JOSM not stable enough (the change in selection code resulted in some bigger errors, some are probably still undetected), but it again gets time to say "please don't checkin anything which takes weeks to fix. :-) So please all developers: We should try to have a new release mid of month (i.e. 2 weeks from now). The help-browser issue in http://josm.openstreetmap.de/ticket/3772 must be fixed before next release. Karl does not really see my rationale behind, but I think in this case I need to pull in my weight as maintainer, so when we don't get this fixed I will revert the external calling code. Ciao -- http://www.dstoecker.eu/ (PGP key available) ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] QuadBuckets in Dataset
On Sat, 2009-10-31 at 18:26 +0100, Jiri Klement wrote:
> > One suggestion was to use the UndoRedoHandler code to catch
> > modifications and reindex the primitives. How about something like
> > this?
>
> I'm not sure if the code will work for one moved node. You first call
> reIndexPrimitives(modified) that might move node to different bucket
> and then try to get ways, that still think the node is in the old
> buckets. Anyway after some modifications it should work, at least as a
> temporary solution.
Yeah, you're right.
One thought I have is that we can have a Map. Before the
executeCommand() happens, we run through all the modified data in
UndoRedoHandler and record all the nodes and their existing coordinates.
After the executeCommand(), we run through the modified nodes again and
compare their coordinates to the ones in the old map. If changed, we
look up any ways using the node, but with the *old* coordinate from the
map.
Another option would be to just give both Nodes and Ways another class
variable that records where they were stuck in the QuadBuckets. If we
ever try to find an existing object, we just use that variable. When we
insert things, we set that variable in the object.
We could either record an old set of coordinates, or we could even put a
pointer directly to the QBLevel object that stores the Way or Node.
remove() in QuadBuckets then becomes a very quick operation that won't
ever fail because an object got moved:
class QuadBuckets {
...
boolean remove(T o) {
QBLevel qb = n.qblevel;
return qb.remove(t);
}
class QBLevel {
...
boolean add_content(T o)
{
boolean ret = false;
if (content == null)
content = new LinkedList();
ret = content.add(o);
if (debug && !this.isLeaf())
pout("added "+o.getClass().getName()+" to non-leaf with
size: " + content.size());
+ o.qblevel = this;
return ret;
}
}
}
The only downside is that it costs an extra pointer.
-- Dave
___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev
[josm-dev] Lean JOSM - one-language trimmed down builds of JOSM
Hi, a while ago I complained about the .jar file having grown to 5.5 GB (meanwhile, over 6 GB), partly due to language support but also due to other kinds of feature creep. Having recently had the pleasure (again) to download JOSM over a slow connection, I decided to do something about this myself, and have set up an alternative JOSM distribution I'm calling "Lean JOSM" for now. Initially I'm just building a few one-language versions (a one-language version is about 3 GB in size, compared to 6 GB of the full jar file). I will start to make my own analysis of "usefulness vs. size" and drop some features that I consider unimportant - as a first start, I have disabled bzip2 reading and writing. Lean JOSM does not have extra Java code - it is just a different way of packing things into a jar file (specifically, some "excludes" and "includes" in the build.xml). Where changes to the code are required to accomodate Lean JOSM, and as long as these don't interfere with normal JOSM operations, I hope to be able to make these changes in the standard JOSM code (as in http://josm.openstreetmap.de/changeset/2367 where I changed the ExtensionFileFilter so that it does not break when bzip2 is not available). I expect that some things I find while trying to slim down the jar file will be useful for standard JOSM as well (like pngcrushing all the image files which I just did). The goal is to create a variant of JOSM that serves 80% of users well but uses significantly less space. Lean JOSM can be downloaded from josm.geofabrik.de. Builds should be roughly in sync with the main JOSM site. There's no "-tested" yet but I plan to introduce that as well. If Lean JOSM works well and becomes popular then we can perhaps move it to the main JOSM site some time. I'm not announcing this on talk yet, I'll just let it run for a while and see how it works. Let me know if you have any comments. If anyone wants a one-language build for their language, I can add that easily (at the moment I'm just doing en and de). Bye Frederik PS: Is it possible to build a JOSM version that supports *only* language X and not English, or is support for English always built-in due to the way we do our I18N? Could I perhaps run a preprocessor to replace the built-in English texts by translations and then kill the whole I18N overhead? -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
