Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi Blake 2016-03-24 16:01 GMT+01:00 Blake Girardot : > > We run into geojson all the time as an interchange format. I can imagine. Reaffirm this at https://josm.openstreetmap.de/ticket/10564 so people believe us. > I was not aware of the plugin somehow, but I just tried it out and it worked > fine for the geojson generated by Field Papers and another community member > reports that she has used it a great deal with Field Papers and it worked > great. Do I understand right: You installed and used succesfully this plugin in JOSM: https://github.com/iandees/josm-geojson ? > I just tried it with a few geojson files generated from JOSM and it does not > seem to import those (example attached) I'm pretty sure, the JOSM export is wrong: This >> "crs":{ "type":"name", "name":"EPSG:4326" }, << should look like this: >> "crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } }, >> :Stefan 2016-03-24 16:01 GMT+01:00 Blake Girardot : > > We run into geojson all the time as an interchange format. > > I was not aware of the plugin somehow, but I just tried it out and it worked > fine for the geojson generated by Field Papers and another community member > reports that she has used it a great deal with Field Papers and it worked > great. > > I just tried it with a few geojson files generated from JOSM and it does not > seem to import those (example attached) > > Cheers, > Blake > > > On 3/6/2016 12:56 AM, Stefan Keller wrote: >> >> To Ian and/or anybody >> >> I'm searching a plugin to import GeoJSON vector data into JOSM. >> >> I of course know Shapefiles but they are deprecated because e.g. they >> cut-off field names at 10 chars. GeoJSON or GeoPackage are better >> alternatives. >> >> Is this plugin still maintained: https://github.com/iandees/josm-geojson ? >> Other plugins or alternatives? >> >> :Stefan >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
We run into geojson all the time as an interchange format. I was not aware of the plugin somehow, but I just tried it out and it worked fine for the geojson generated by Field Papers and another community member reports that she has used it a great deal with Field Papers and it worked great. I just tried it with a few geojson files generated from JOSM and it does not seem to import those (example attached) Cheers, Blake On 3/6/2016 12:56 AM, Stefan Keller wrote: To Ian and/or anybody I'm searching a plugin to import GeoJSON vector data into JOSM. I of course know Shapefiles but they are deprecated because e.g. they cut-off field names at 10 chars. GeoJSON or GeoPackage are better alternatives. Is this plugin still maintained: https://github.com/iandees/josm-geojson ? Other plugins or alternatives? :Stefan ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk { "type":"FeatureCollection", "crs":{ "type":"name", "name":"EPSG:4326" }, "generator":"JOSM", "features":[ { "type":"Feature", "properties":{ }, "geometry":{ "type":"LineString", "coordinates":[ [ 34.39455, -21.23088 ], [ 34.11719, -21.06965 ], [ 34.10904, -21.10852 ], [ 34.089979, -21.12866 ], [ 34.05608, -21.1497195 ], [ 34.03801, -21.17419 ], [ 34.03241, -21.2312798 ], [ 33.9835372, -21.3199553 ], [ 34.0085905, -21.31264 ], [ 34.0501294, -21.3036911 ], [ 34.08934, -21.2932803 ], [ 34.13111, -21.30658 ], [ 34.1481282, -21.3261484 ], [ 34.18078, -21.3154598 ], [ 34.21064, -21.29569 ], [ 34.221319, -21.27627 ], [ 34.22472, -21.2726097 ], [ 34.25293, -21.27256 ], [ 34.27975, -21.2883405 ], [ 34.308941, -21.3120304 ], [ 34.3301868, -21.3177579 ], [ 34.34999, -21.2902803 ], [ 34.38400864274337, -21.262989190020335 ], [ 34.39455, -21.23088 ] ] } } ] }___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi Frederik and Jukka Before I try give answers to performance let's be aware that we're (at least I am) speaking about a "desktop exchange format", not a storage fomat for GIS processing. But Frederik's comment piqued my curiosity and I did some quick comparison. I generated 1 mio. records in PostsGIS with this table CREATE TABLE benchmark (id serial primary key, txt varchar(32), geom geometry(point,4326) ); Then I used OGR2OGR to create the following three file formats: GeoPackage (using 73.9 MB disk space), Shapefiles (dbf/shp/shx 117 MB) and Spatialite (173 MB). Creation time of GeoPackage was 18 sec., Shapefile 21 sec. and Spatialite 1 min 51 sec. So, GeoPackage is a bit faster than Shapefiles and significantly (about 37%) smaller in size. Spatialite in fact consumes much more disk space than Shapefile and GeoPackage, and Spatialite is several times slower for creation time. This could explain the preformance issues of Spatialite Frederic mentioned. :Stefan 2016-03-22 13:56 GMT+01:00 Jukka Rahkonen : > Frederik Ramm remote.org> writes: > >> >> Hi, >> >> On 03/20/2016 10:56 PM, Stefan Keller wrote: >> > But Shapefile remains an oldtimer with more drawbacks than limited >> > field names; see [1]. >> > GeoJSON (ascii) and GeoPackages (binary) are formats which are more >> > suited for the job. >> > I still have hope that JOSM will be able to read those vector formats too. >> >> Frankly, whenever I venture into the brave new world of Spatialite, I >> come back to good old shape files after a while for performance reasons. >> I'm not sure if Geopackage has significant performance improvements over >> simple Spatialite but if it hasn't then my recommendation for simple GIS >> processing is certainly to stick with shape files for the time being - >> despite all their shortcomings. > > > Hi Frederic, > > I would like to receive some sample data, exact way to reproduce some of > your ventures and cold numbers about the speed you have experienced. > Spatialite does have it's limits but for plain selects with spatial and > attribute filters it can well outperform both shapefiles and PostGIS. > > I keep most vector data for WMS services in Spatialite or GeoPackage due to > the already mentioned and some other reasons: > - supports long attribute names > - supports strings longer than 255 characters > - supports SQL > - supports attribute indexes > - much less encoding problems due to UTF-8 > - one single file vs. a bunch of files in shapefile, perhaps even split to > separate bunches for points, lines and polygons. > > For me SpatiaLite is a little bit slower than shapefiles if only spatial > filter (BBOX) is used but usually faster if also attribute filters are > involved, especially if more than one field is needed in filters (Shapefiles > can be sorted by one attribute only). Of course spatialite must have indexes > which suit the queries and when it comes to spatial index, the client must > know how to utilize the table based R-Tree index. I also recommend to VACUUM > once the database is ready to use. > > Many spatial operations are relatively slow in Spatialite and I don't > usually utilize them on-the-fly with WMS server. Instead, I run the > algorithm once and store the result into a new table because a few > mega/gigabytes of additional disk space is not crucial on the server. > However, such operations tend to be slow also if shapefiles are used as > source data. > > Write performance especially with concurrent writes is another story. I am > talking about read-only operations. I know that I am writing empty words as > far as I do not include reproducible facts but I am willing to join to a > controlled test if someone is organizing such. > > -Jukka Rahkonen- > > > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Frederik Ramm remote.org> writes: > > Hi, > > On 03/20/2016 10:56 PM, Stefan Keller wrote: > > But Shapefile remains an oldtimer with more drawbacks than limited > > field names; see [1]. > > GeoJSON (ascii) and GeoPackages (binary) are formats which are more > > suited for the job. > > I still have hope that JOSM will be able to read those vector formats too. > > Frankly, whenever I venture into the brave new world of Spatialite, I > come back to good old shape files after a while for performance reasons. > I'm not sure if Geopackage has significant performance improvements over > simple Spatialite but if it hasn't then my recommendation for simple GIS > processing is certainly to stick with shape files for the time being - > despite all their shortcomings. Hi Frederic, I would like to receive some sample data, exact way to reproduce some of your ventures and cold numbers about the speed you have experienced. Spatialite does have it's limits but for plain selects with spatial and attribute filters it can well outperform both shapefiles and PostGIS. I keep most vector data for WMS services in Spatialite or GeoPackage due to the already mentioned and some other reasons: - supports long attribute names - supports strings longer than 255 characters - supports SQL - supports attribute indexes - much less encoding problems due to UTF-8 - one single file vs. a bunch of files in shapefile, perhaps even split to separate bunches for points, lines and polygons. For me SpatiaLite is a little bit slower than shapefiles if only spatial filter (BBOX) is used but usually faster if also attribute filters are involved, especially if more than one field is needed in filters (Shapefiles can be sorted by one attribute only). Of course spatialite must have indexes which suit the queries and when it comes to spatial index, the client must know how to utilize the table based R-Tree index. I also recommend to VACUUM once the database is ready to use. Many spatial operations are relatively slow in Spatialite and I don't usually utilize them on-the-fly with WMS server. Instead, I run the algorithm once and store the result into a new table because a few mega/gigabytes of additional disk space is not crucial on the server. However, such operations tend to be slow also if shapefiles are used as source data. Write performance especially with concurrent writes is another story. I am talking about read-only operations. I know that I am writing empty words as far as I do not include reproducible facts but I am willing to join to a controlled test if someone is organizing such. -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
On Mon, Mar 21, 2016 at 1:53 PM, Christoph Hormann wrote: > > The general problem with SQLite based files is that they are always read > and written through the SQLite database engine which is not > particularly good for sequential write and read operations. In my experience the engine defaults to one write operation per transaction which is very slow. If you are processing a large amount of data you need to explicitly open and then later (after many writes) commit the transaction if you want reasonable performance. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
On Monday 21 March 2016, Stefan Keller wrote: > > I'm not sure if Geopackage has significant performance improvements > > over simple Spatialite but if it hasn't then my recommendation for > > simple GIS processing is certainly to stick with shape files for > > the time being - despite all their shortcomings. > > Shapefiles, Spatialite and GeoPackage all use R-tree indexes. > I never heard about performance advantages of Shapefiles (except a > bug in a former Spatialite driver). The general problem with SQLite based files is that they are always read and written through the SQLite database engine which is not particularly good for sequential write and read operations. You can get reasonable performance from Spatialite in many cases but you never get anywhere close to shapefiles in my experience. > Can you tell me a use case? Which Spatialite SW are you using? I am not aware of any software dealing with spatialite files that does not use libspatialite/SQLite. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi, 2016-03-21 17:33 GMT+01:00 Frederik Ramm : > I'm not sure if Geopackage has significant performance improvements over > simple Spatialite but if it hasn't then my recommendation for simple GIS > processing is certainly to stick with shape files for the time being - > despite all their shortcomings. Shapefiles, Spatialite and GeoPackage all use R-tree indexes. I never heard about performance advantages of Shapefiles (except a bug in a former Spatialite driver). Can you tell me a use case? Which Spatialite SW are you using? What people need to realize is, that Shapefiles really have issues: 1. Attribute name length restricted to 10 chars => resulting in information loss. 2. Strings length restricted to 255 chars => resulting in information loss. 3. No Multi LineString/Multi Polygon => resulting in information loss. 4. Max. file size restricted to 2GB => resulting in splitting tables into several files. 5. Several cluttered files => makes users forget to include all. :Stefan 2016-03-21 17:33 GMT+01:00 Frederik Ramm : > Hi, > > On 03/20/2016 10:56 PM, Stefan Keller wrote: >> But Shapefile remains an oldtimer with more drawbacks than limited >> field names; see [1]. >> GeoJSON (ascii) and GeoPackages (binary) are formats which are more >> suited for the job. >> I still have hope that JOSM will be able to read those vector formats too. > > Frankly, whenever I venture into the brave new world of Spatialite, I > come back to good old shape files after a while for performance reasons. > I'm not sure if Geopackage has significant performance improvements over > simple Spatialite but if it hasn't then my recommendation for simple GIS > processing is certainly to stick with shape files for the time being - > despite all their shortcomings. > > Bye > Frederik > > -- > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi, On 03/20/2016 10:56 PM, Stefan Keller wrote: > But Shapefile remains an oldtimer with more drawbacks than limited > field names; see [1]. > GeoJSON (ascii) and GeoPackages (binary) are formats which are more > suited for the job. > I still have hope that JOSM will be able to read those vector formats too. Frankly, whenever I venture into the brave new world of Spatialite, I come back to good old shape files after a while for performance reasons. I'm not sure if Geopackage has significant performance improvements over simple Spatialite but if it hasn't then my recommendation for simple GIS processing is certainly to stick with shape files for the time being - despite all their shortcomings. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
His Stefan Nice hack! But Shapefile remains an oldtimer with more drawbacks than limited field names; see [1]. GeoJSON (ascii) and GeoPackages (binary) are formats which are more suited for the job. I still have hope that JOSM will be able to read those vector formats too. :Stefan [1] http://giswiki.hsr.ch/Shapefile 2016-03-20 19:29 GMT+01:00 Stefan Baebler : > Hi! > > On Sun, Mar 6, 2016 at 12:56 AM, Stefan Keller wrote: >> >> I of course know Shapefiles but they are deprecated because e.g. they >> cut-off field names at 10 chars. GeoJSON or GeoPackage are better >> alternatives. > > > For RABA-KGZ landuse import in Slovenia we pushed the-10 character limit in > shapefiles / dbf to 11 characters and even inserted special characters > (colons) in field name (to allow us to have "source:date" tag) by altering > the binary data using bbe (sed-like binary block editor) as the last step of > data preparation: > bbe -e "s/SOURCEDATE\x00/source:date/" source.dbf -o target.dbf > > JOSM reads it nicely using OpenData plugin, as well as QGIS and other > editors. > > Full example and context can be seen at > https://github.com/stefanb/RabaSplitForOSM/blob/master/makeOneSplitExplode.sh#L94 > > Example shapefiles can be seen (in browser and in JOSM) at > http://raba.openstreetmap.si > > best regards, > Stefan ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi! On Sun, Mar 6, 2016 at 12:56 AM, Stefan Keller wrote: > > I of course know Shapefiles but they are deprecated because e.g. they > cut-off field names at 10 chars. GeoJSON or GeoPackage are better > alternatives. > For RABA-KGZ landuse import in Slovenia we pushed the-10 character limit in shapefiles / dbf to 11 characters and even inserted special characters (colons) in field name (to allow us to have "source:date" tag) by altering the binary data using bbe (sed-like binary block editor) as the last step of data preparation: bbe -e "s/SOURCEDATE\x00/source:date/" source.dbf -o target.dbf JOSM reads it nicely using OpenData plugin, as well as QGIS and other editors. Full example and context can be seen at https://github.com/stefanb/RabaSplitForOSM/blob/master/makeOneSplitExplode.sh#L94 Example shapefiles can be seen (in browser and in JOSM) at http://raba.openstreetmap.si best regards, Stefan ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Hi Jan Many thanks for this status update of the GeoJSON plugin code and to ogr2osm. My use case is that I have to postprocess admin. boundaries from the cadastre/topogr. bureau in order to produce a background layer (to JOSM) which indicates boundaries which have been deleted/moved/added. :Stefan P.S. @Felix: Thanks too for your answer referring to the "opendata" plugin (weird name for a GIS format importer). I'd say, ignoring GeoJSON won't help (JOSM) neither. But that's another issue. 2016-03-07 15:37 GMT+01:00 Ian Dees : > On Sat, Mar 5, 2016 at 6:56 PM, Stefan Keller wrote: >> >> To Ian and/or anybody >> >> I'm searching a plugin to import GeoJSON vector data into JOSM. >> >> I of course know Shapefiles but they are deprecated because e.g. they >> cut-off field names at 10 chars. GeoJSON or GeoPackage are better >> alternatives. >> >> Is this plugin still maintained: https://github.com/iandees/josm-geojson ? >> Other plugins or alternatives? > > > I started looking at this again last week but didn't make much progress. I > would second Jukka's suggestion of ogr2osm. It will convert the GeoJSON into > useful OSM files rather quickly and painlessly. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
On Sat, Mar 5, 2016 at 6:56 PM, Stefan Keller wrote: > To Ian and/or anybody > > I'm searching a plugin to import GeoJSON vector data into JOSM. > > I of course know Shapefiles but they are deprecated because e.g. they > cut-off field names at 10 chars. GeoJSON or GeoPackage are better > alternatives. > > Is this plugin still maintained: https://github.com/iandees/josm-geojson ? > Other plugins or alternatives? I started looking at this again last week but didn't make much progress. I would second Jukka's suggestion of ogr2osm. It will convert the GeoJSON into useful OSM files rather quickly and painlessly. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
Stefan Keller gmail.com> writes: > > To Ian and/or anybody > > I'm searching a plugin to import GeoJSON vector data into JOSM. > > I of course know Shapefiles but they are deprecated because e.g. they > cut-off field names at 10 chars. GeoJSON or GeoPackage are better > alternatives. > > Is this plugin still maintained: https://github.com/iandees/josm-geojson ? > Other plugins or alternatives? Perhaps ogr2osm still works http://wiki.openstreetmap.org/wiki/Ogr2osm -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] JOSM plugin to import GeoJSON?
This won't help. But to give yoy some context: There is a "won't fix" request ticket for JOSM to support GeoJSON: https://josm.openstreetmap.de/ticket/10564 Best, Felix On 06/03/16 00:56, Stefan Keller wrote: > To Ian and/or anybody > > I'm searching a plugin to import GeoJSON vector data into JOSM. > > I of course know Shapefiles but they are deprecated because e.g. they > cut-off field names at 10 chars. GeoJSON or GeoPackage are better > alternatives. > > Is this plugin still maintained: https://github.com/iandees/josm-geojson ? > Other plugins or alternatives? > > :Stefan > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk