Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Keller]

Hi Blake

2016-03-24 16:01 GMT+01:00 Blake Girardot :
>
> We run into geojson all the time as an interchange format.

I can imagine. Reaffirm this at
https://josm.openstreetmap.de/ticket/10564 so people believe us.


> I was not aware of the plugin somehow, but I just tried it out and it worked
> fine for the geojson generated by Field Papers and another community member
> reports that she has used it a great deal with Field Papers and it worked
> great.

Do I understand right: You installed and used succesfully this plugin
in JOSM: https://github.com/iandees/josm-geojson ?

> I just tried it with a few geojson files generated from JOSM and it does not
> seem to import those (example attached)

I'm pretty sure, the JOSM export is wrong:

This
>>
"crs":{
"type":"name",
"name":"EPSG:4326"
},
<<

should look like this:
>>
"crs": {
"type": "name",
"properties": {
  "name": "urn:ogc:def:crs:OGC:1.3:CRS84"
}
  },
>>

:Stefan


2016-03-24 16:01 GMT+01:00 Blake Girardot :
>
> We run into geojson all the time as an interchange format.
>
> I was not aware of the plugin somehow, but I just tried it out and it worked
> fine for the geojson generated by Field Papers and another community member
> reports that she has used it a great deal with Field Papers and it worked
> great.
>
> I just tried it with a few geojson files generated from JOSM and it does not
> seem to import those (example attached)
>
> Cheers,
> Blake
>
>
> On 3/6/2016 12:56 AM, Stefan Keller wrote:
>>
>> To Ian and/or anybody
>>
>> I'm searching a plugin to import GeoJSON vector data into JOSM.
>>
>> I of course know Shapefiles but they are deprecated because e.g. they
>> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
>> alternatives.
>>
>> Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
>> Other plugins or alternatives?
>>
>> :Stefan
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Blake Girardot]

We run into geojson all the time as an interchange format.

I was not aware of the plugin somehow, but I just tried it out and it 
worked fine for the geojson generated by Field Papers and another 
community member reports that she has used it a great deal with Field 
Papers and it worked great.


I just tried it with a few geojson files generated from JOSM and it does 
not seem to import those (example attached)


Cheers,
Blake

On 3/6/2016 12:56 AM, Stefan Keller wrote:

To Ian and/or anybody

I'm searching a plugin to import GeoJSON vector data into JOSM.

I of course know Shapefiles but they are deprecated because e.g. they
cut-off field names at 10 chars. GeoJSON or GeoPackage are better
alternatives.

Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
Other plugins or alternatives?

:Stefan

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


{
"type":"FeatureCollection",
"crs":{
"type":"name",
"name":"EPSG:4326"
},
"generator":"JOSM",
"features":[
{
"type":"Feature",
"properties":{
},
"geometry":{
"type":"LineString",
"coordinates":[
[
34.39455,
-21.23088
],
[
34.11719,
-21.06965
],
[
34.10904,
-21.10852
],
[
34.089979,
-21.12866
],
[
34.05608,
-21.1497195
],
[
34.03801,
-21.17419
],
[
34.03241,
-21.2312798
],
[
33.9835372,
-21.3199553
],
[
34.0085905,
-21.31264
],
[
34.0501294,
-21.3036911
],
[
34.08934,
-21.2932803
],
[
34.13111,
-21.30658
],
[
34.1481282,
-21.3261484
],
[
34.18078,
-21.3154598
],
[
34.21064,
-21.29569
],
[
34.221319,
-21.27627
],
[
34.22472,
-21.2726097
],
[
34.25293,
-21.27256
],
[
34.27975,
-21.2883405
],
[
34.308941,
-21.3120304
],
[
34.3301868,
-21.3177579
],
[
34.34999,
-21.2902803
],
[
34.38400864274337,
-21.262989190020335
],
[
34.39455,
-21.23088
]
]
}
}
]
}___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Keller]

Hi Frederik and Jukka

Before I try give answers to performance let's be aware that we're (at
least I am) speaking about a "desktop exchange format", not a storage
fomat for GIS processing.

But Frederik's comment piqued my curiosity and I did some quick comparison.
I generated 1 mio. records in PostsGIS with this table
CREATE TABLE benchmark (id serial primary key, txt varchar(32), geom
geometry(point,4326) );

Then I used OGR2OGR to create the following three file formats:
GeoPackage (using 73.9 MB disk space), Shapefiles (dbf/shp/shx 117 MB)
and Spatialite (173 MB).

Creation time of GeoPackage was 18 sec., Shapefile 21 sec. and
Spatialite 1 min 51 sec.
So, GeoPackage is a bit faster than Shapefiles and significantly
(about 37%) smaller in size.
Spatialite in fact consumes much more disk space than Shapefile and
GeoPackage, and Spatialite is several times slower for creation time.

This could explain the preformance issues of Spatialite Frederic mentioned.

:Stefan

2016-03-22 13:56 GMT+01:00 Jukka Rahkonen :
> Frederik Ramm  remote.org> writes:
>
>>
>> Hi,
>>
>> On 03/20/2016 10:56 PM, Stefan Keller wrote:
>> > But Shapefile remains an oldtimer with more drawbacks than limited
>> > field names; see [1].
>> > GeoJSON (ascii) and GeoPackages (binary) are formats which are more
>> > suited for the job.
>> > I still have hope that JOSM will be able to read those vector formats too.
>>
>> Frankly, whenever I venture into the brave new world of Spatialite, I
>> come back to good old shape files after a while for performance reasons.
>> I'm not sure if Geopackage has significant performance improvements over
>> simple Spatialite but if it hasn't then my recommendation for simple GIS
>> processing is certainly to stick with shape files for the time being -
>> despite all their shortcomings.
>
>
> Hi Frederic,
>
> I would like to receive some sample data, exact way to reproduce some of
> your ventures and cold numbers about the speed you have experienced.
> Spatialite does have it's limits but for plain selects with spatial and
> attribute filters it can well outperform both shapefiles and PostGIS.
>
> I keep most vector data for WMS services in Spatialite or GeoPackage due to
> the already mentioned and some other reasons:
> - supports long attribute names
> - supports strings longer than 255 characters
> - supports SQL
> - supports attribute indexes
> - much less encoding problems due to UTF-8
> - one single file vs. a bunch of files in shapefile, perhaps even split to
> separate bunches for points, lines and polygons.
>
> For me SpatiaLite is a little bit slower than shapefiles if only spatial
> filter (BBOX) is used but usually faster if also attribute filters are
> involved, especially if more than one field is needed in filters (Shapefiles
> can be sorted by one attribute only). Of course spatialite must have indexes
> which suit the queries and when it comes to spatial index, the client must
> know how to utilize the table based R-Tree index. I also recommend to VACUUM
> once the database is ready to use.
>
> Many spatial operations are relatively slow in Spatialite and I don't
> usually utilize them on-the-fly with WMS server. Instead, I run the
> algorithm once and store the result into a new table because a few
> mega/gigabytes of additional disk space is not crucial on the server.
> However, such operations tend to be slow also if shapefiles are used as
> source data.
>
> Write performance especially with concurrent writes is another story. I am
> talking about read-only operations. I know that I am writing empty words as
> far as I do not include reproducible facts but I am willing to join to a
> controlled test if someone is organizing such.
>
> -Jukka Rahkonen-
>
>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Jukka Rahkonen]

Frederik Ramm  remote.org> writes:

> 
> Hi,
> 
> On 03/20/2016 10:56 PM, Stefan Keller wrote:
> > But Shapefile remains an oldtimer with more drawbacks than limited
> > field names; see [1].
> > GeoJSON (ascii) and GeoPackages (binary) are formats which are more
> > suited for the job.
> > I still have hope that JOSM will be able to read those vector formats too.
> 
> Frankly, whenever I venture into the brave new world of Spatialite, I
> come back to good old shape files after a while for performance reasons.
> I'm not sure if Geopackage has significant performance improvements over
> simple Spatialite but if it hasn't then my recommendation for simple GIS
> processing is certainly to stick with shape files for the time being -
> despite all their shortcomings.


Hi Frederic,

I would like to receive some sample data, exact way to reproduce some of
your ventures and cold numbers about the speed you have experienced.
Spatialite does have it's limits but for plain selects with spatial and
attribute filters it can well outperform both shapefiles and PostGIS. 

I keep most vector data for WMS services in Spatialite or GeoPackage due to
the already mentioned and some other reasons:
- supports long attribute names
- supports strings longer than 255 characters
- supports SQL
- supports attribute indexes
- much less encoding problems due to UTF-8
- one single file vs. a bunch of files in shapefile, perhaps even split to
separate bunches for points, lines and polygons.

For me SpatiaLite is a little bit slower than shapefiles if only spatial
filter (BBOX) is used but usually faster if also attribute filters are
involved, especially if more than one field is needed in filters (Shapefiles
can be sorted by one attribute only). Of course spatialite must have indexes
which suit the queries and when it comes to spatial index, the client must
know how to utilize the table based R-Tree index. I also recommend to VACUUM
once the database is ready to use.

Many spatial operations are relatively slow in Spatialite and I don't
usually utilize them on-the-fly with WMS server. Instead, I run the
algorithm once and store the result into a new table because a few
mega/gigabytes of additional disk space is not crucial on the server. 
However, such operations tend to be slow also if shapefiles are used as
source data.

Write performance especially with concurrent writes is another story. I am
talking about read-only operations. I know that I am writing empty words as
far as I do not include reproducible facts but I am willing to join to a
controlled test if someone is organizing such.

-Jukka Rahkonen-




___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Mike Thompson]

On Mon, Mar 21, 2016 at 1:53 PM, Christoph Hormann 
wrote:

>
> The general problem with SQLite based files is that they are always read
> and written through the SQLite database engine which is not
> particularly good for sequential write and read operations.

In my experience the engine defaults to one write operation per transaction
which is very slow.  If you are processing a large amount of data you need
to explicitly open and then later (after many writes) commit the
transaction if you want reasonable performance.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Christoph Hormann]

On Monday 21 March 2016, Stefan Keller wrote:
> > I'm not sure if Geopackage has significant performance improvements
> > over simple Spatialite but if it hasn't then my recommendation for
> > simple GIS processing is certainly to stick with shape files for
> > the time being - despite all their shortcomings.
>
> Shapefiles, Spatialite and GeoPackage all use R-tree indexes.
> I never heard about performance advantages of Shapefiles (except a
> bug in a former Spatialite driver).

The general problem with SQLite based files is that they are always read 
and written through the SQLite database engine which is not 
particularly good for sequential write and read operations.  You can 
get reasonable performance from Spatialite in many cases but you never 
get anywhere close to shapefiles in my experience.

> Can you tell me a use case? Which Spatialite SW are you using?

I am not aware of any software dealing with spatialite files that does 
not use libspatialite/SQLite.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Keller]

Hi,

2016-03-21 17:33 GMT+01:00 Frederik Ramm :
> I'm not sure if Geopackage has significant performance improvements over
> simple Spatialite but if it hasn't then my recommendation for simple GIS
> processing is certainly to stick with shape files for the time being -
> despite all their shortcomings.

Shapefiles, Spatialite and GeoPackage all use R-tree indexes.
I never heard about performance advantages of Shapefiles (except a bug
in a former Spatialite driver).
Can you tell me a use case? Which Spatialite SW are you using?

What people need to realize is, that Shapefiles really have issues:
1. Attribute name length restricted to 10 chars => resulting in
information loss.
2. Strings length restricted to 255 chars => resulting in information loss.
3. No Multi LineString/Multi Polygon => resulting in information loss.
4. Max. file size restricted to 2GB => resulting in splitting tables
into several files.
5. Several cluttered files => makes users forget to include all.

:Stefan


2016-03-21 17:33 GMT+01:00 Frederik Ramm :
> Hi,
>
> On 03/20/2016 10:56 PM, Stefan Keller wrote:
>> But Shapefile remains an oldtimer with more drawbacks than limited
>> field names; see [1].
>> GeoJSON (ascii) and GeoPackages (binary) are formats which are more
>> suited for the job.
>> I still have hope that JOSM will be able to read those vector formats too.
>
> Frankly, whenever I venture into the brave new world of Spatialite, I
> come back to good old shape files after a while for performance reasons.
> I'm not sure if Geopackage has significant performance improvements over
> simple Spatialite but if it hasn't then my recommendation for simple GIS
> processing is certainly to stick with shape files for the time being -
> despite all their shortcomings.
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Frederik Ramm]

Hi,

On 03/20/2016 10:56 PM, Stefan Keller wrote:
> But Shapefile remains an oldtimer with more drawbacks than limited
> field names; see [1].
> GeoJSON (ascii) and GeoPackages (binary) are formats which are more
> suited for the job.
> I still have hope that JOSM will be able to read those vector formats too.

Frankly, whenever I venture into the brave new world of Spatialite, I
come back to good old shape files after a while for performance reasons.
I'm not sure if Geopackage has significant performance improvements over
simple Spatialite but if it hasn't then my recommendation for simple GIS
processing is certainly to stick with shape files for the time being -
despite all their shortcomings.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Keller]

His Stefan

Nice hack!
But Shapefile remains an oldtimer with more drawbacks than limited
field names; see [1].
GeoJSON (ascii) and GeoPackages (binary) are formats which are more
suited for the job.
I still have hope that JOSM will be able to read those vector formats too.

:Stefan

[1] http://giswiki.hsr.ch/Shapefile


2016-03-20 19:29 GMT+01:00 Stefan Baebler :
> Hi!
>
> On Sun, Mar 6, 2016 at 12:56 AM, Stefan Keller  wrote:
>>
>> I of course know Shapefiles but they are deprecated because e.g. they
>> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
>> alternatives.
>
>
> For RABA-KGZ landuse import in Slovenia we pushed the-10 character limit in
> shapefiles / dbf to 11 characters and even inserted special characters
> (colons) in field name (to allow us to have "source:date" tag) by altering
> the binary data using bbe (sed-like binary block editor) as the last step of
> data preparation:
> bbe -e "s/SOURCEDATE\x00/source:date/" source.dbf -o target.dbf
>
> JOSM reads it nicely using OpenData plugin, as well as QGIS and other
> editors.
>
> Full example and context can be seen at
> https://github.com/stefanb/RabaSplitForOSM/blob/master/makeOneSplitExplode.sh#L94
>
> Example shapefiles can be seen (in browser and in JOSM) at
> http://raba.openstreetmap.si
>
> best regards,
> Stefan

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Baebler]

Hi!

On Sun, Mar 6, 2016 at 12:56 AM, Stefan Keller  wrote:
>
> I of course know Shapefiles but they are deprecated because e.g. they
> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
> alternatives.
>

For RABA-KGZ landuse import in Slovenia we pushed the-10 character limit in
shapefiles / dbf to 11 characters and even inserted special characters
(colons) in field name (to allow us to have "source:date" tag) by altering
the binary data using bbe (sed-like binary block editor) as the last step
of data preparation:
bbe -e "s/SOURCEDATE\x00/source:date/" source.dbf -o target.dbf

JOSM reads it nicely using OpenData plugin, as well as QGIS and other
editors.

Full example and context can be seen at
https://github.com/stefanb/RabaSplitForOSM/blob/master/makeOneSplitExplode.sh#L94

Example shapefiles can be seen (in browser and in JOSM) at
http://raba.openstreetmap.si

best regards,
Stefan
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Stefan Keller]

Hi Jan

Many thanks for this status update of the GeoJSON plugin code and to ogr2osm.
My use case is that I have to postprocess admin. boundaries from the
cadastre/topogr. bureau in order to produce a background layer (to
JOSM) which indicates boundaries which have been deleted/moved/added.

:Stefan

P.S. @Felix: Thanks too for your answer referring to the "opendata"
plugin (weird name for a GIS format importer). I'd say, ignoring
GeoJSON won't help (JOSM) neither. But that's another issue.


2016-03-07 15:37 GMT+01:00 Ian Dees :
> On Sat, Mar 5, 2016 at 6:56 PM, Stefan Keller  wrote:
>>
>> To Ian and/or anybody
>>
>> I'm searching a plugin to import GeoJSON vector data into JOSM.
>>
>> I of course know Shapefiles but they are deprecated because e.g. they
>> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
>> alternatives.
>>
>> Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
>> Other plugins or alternatives?
>
>
> I started looking at this again last week but didn't make much progress. I
> would second Jukka's suggestion of ogr2osm. It will convert the GeoJSON into
> useful OSM files rather quickly and painlessly.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Ian Dees]

On Sat, Mar 5, 2016 at 6:56 PM, Stefan Keller  wrote:

> To Ian and/or anybody
>
> I'm searching a plugin to import GeoJSON vector data into JOSM.
>
> I of course know Shapefiles but they are deprecated because e.g. they
> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
> alternatives.
>
> Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
> Other plugins or alternatives?


I started looking at this again last week but didn't make much progress. I
would second Jukka's suggestion of ogr2osm. It will convert the GeoJSON
into useful OSM files rather quickly and painlessly.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Jukka Rahkonen]

Stefan Keller  gmail.com> writes:

> 
> To Ian and/or anybody
> 
> I'm searching a plugin to import GeoJSON vector data into JOSM.
> 
> I of course know Shapefiles but they are deprecated because e.g. they
> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
> alternatives.
> 
> Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
> Other plugins or alternatives?

Perhaps ogr2osm still works http://wiki.openstreetmap.org/wiki/Ogr2osm

-Jukka Rahkonen-





___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] JOSM plugin to import GeoJSON?

[Felix Delattre]

This won't help. But to give yoy some context: There is a "won't fix"
request ticket for JOSM to support GeoJSON:
https://josm.openstreetmap.de/ticket/10564

Best,
Felix


On 06/03/16 00:56, Stefan Keller wrote:
> To Ian and/or anybody
>
> I'm searching a plugin to import GeoJSON vector data into JOSM.
>
> I of course know Shapefiles but they are deprecated because e.g. they
> cut-off field names at 10 chars. GeoJSON or GeoPackage are better
> alternatives.
>
> Is this plugin still maintained: https://github.com/iandees/josm-geojson ?
> Other plugins or alternatives?
>
> :Stefan
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk