Re: [OSM-talk] recommendation for JSON to CSV converter

[Florian Lohoff]

On Fri, Mar 01, 2024 at 03:23:29PM +0100, Martin Trautmann wrote:
> On 01.03.24 14:52, Florian Lohoff wrote:
> 
> > For obvious reasons Essen cant have 2.5 or even 4.4 Mio addresses.
> 
> Yeah, I had noticed by now that this is Essen only. I don't find any
> json info for all of NRW, from this year.

NRW has not updated their Open Data as they are in the middle of a
Data Format switch. I asked them twice already 

They havent updated "Hausumringe" aswell.

Flo
-- 
Florian Lohoff f...@zz.de
  Any sufficiently advanced technology is indistinguishable from magic.


signature.asc
Description: PGP signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Martin Trautmann via talk]

On 01.03.24 14:52, Florian Lohoff wrote:

> For obvious reasons Essen cant have 2.5 or even 4.4 Mio addresses.

Yeah, I had noticed by now that this is Essen only. I don't find any
json info for all of NRW, from this year.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Florian Lohoff]

On Thu, Feb 29, 2024 at 10:17:28PM +0100, Martin Trautmann wrote:
> On 28.02.24 22:58, Florian Lohoff wrote:
> 
> > For these kind of simple conversions there is no need for programming.
> >
> > Its a GeoJSON Featurecollection. So a list of GeoJSON features,
> > each of them having a "properties" objects with all the attributes
> >
> > jq -r ".features[] | 
> > [.properties.Strassenna,.properties.HsNr_Zus,.properties.Xg,.properties.Yg] 
> > | @csv" Hauskoordinaten_0822.geojson
> > "Stinnesstr.","8",7.010088,51.525567
> > "Timpestr.","29",7.010417,51.522377
> > "Hattramstr.","31",7.003345,51.52098
> > "Spakenbroich","69",7.003248,51.523044
> > "Bertramstr.","36",7.005647,51.522393
> > "Bertramstr.","38",7.005608,51.522429
> > "Bertramstr.","40",7.005566,51.522469
> > [...]
> 
> Thanks, that worked pretty well. So I'll have a closer Look at jq now.
> 
> I'll have to figure out why the geojson contains 99 606 records only,
> although the former version contains 4 418 144 records

Most likely lines, not records. A JSON objects spans a lot of rows so
counting rows is the issue.

2.5 Mio lines, 99606 Features, 1 Featurecollection.

flo@p5:~$ wc -l Hauskoordinaten_0822.geojson 
2589766 Hauskoordinaten_0822.geojson
flo@p5:~$ rgrep Feature Hauskoordinaten_0822.geojson  | wc -l
99607

For obvious reasons Essen cant have 2.5 or even 4.4 Mio addresses.

Flo
-- 
Florian Lohoff f...@zz.de
  Any sufficiently advanced technology is indistinguishable from magic.


signature.asc
Description: PGP signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Martin Trautmann via talk]

On 28.02.24 22:58, Florian Lohoff wrote:

> For these kind of simple conversions there is no need for programming.
>
> Its a GeoJSON Featurecollection. So a list of GeoJSON features,
> each of them having a "properties" objects with all the attributes
>
> jq -r ".features[] | 
> [.properties.Strassenna,.properties.HsNr_Zus,.properties.Xg,.properties.Yg] | 
> @csv" Hauskoordinaten_0822.geojson
> "Stinnesstr.","8",7.010088,51.525567
> "Timpestr.","29",7.010417,51.522377
> "Hattramstr.","31",7.003345,51.52098
> "Spakenbroich","69",7.003248,51.523044
> "Bertramstr.","36",7.005647,51.522393
> "Bertramstr.","38",7.005608,51.522429
> "Bertramstr.","40",7.005566,51.522469
> [...]

Thanks, that worked pretty well. So I'll have a closer Look at jq now.

I'll have to figure out why the geojson contains 99 606 records only,
although the former version contains 4 418 144 records


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Florian Lohoff]

Hola,

On Wed, Feb 28, 2024 at 04:16:24PM +0100, Martin Trautmann via talk wrote:
> Is there any JSON converter that you would recommend?
> 
> There are plenty of web services, but none of those worked for me, on
> 
> (70 MB)
> 
> There are plenty of starter tutorials, in order to use python,
> javascript etc.
> 
> But those did not work either.
> 
> I tried dasel, without success (dasel version v2.5.0)
> 
> So I wonder what the best solution might be, apart of writing my own
> converter (probably in perl).
> 
> What I would expect is hopefully a 100 % conversion - or an error log
> which data could not be processed.
> 
> The problem here seems to be the nesting of several levels.

For these kind of simple conversions there is no need for programming.

Its a GeoJSON Featurecollection. So a list of GeoJSON features,
each of them having a "properties" objects with all the attributes

jq -r ".features[] | 
[.properties.Strassenna,.properties.HsNr_Zus,.properties.Xg,.properties.Yg] | 
@csv" Hauskoordinaten_0822.geojson
"Stinnesstr.","8",7.010088,51.525567
"Timpestr.","29",7.010417,51.522377
"Hattramstr.","31",7.003345,51.52098
"Spakenbroich","69",7.003248,51.523044
"Bertramstr.","36",7.005647,51.522393
"Bertramstr.","38",7.005608,51.522429
"Bertramstr.","40",7.005566,51.522469
[...]

Flo
-- 
Florian Lohoff f...@zz.de
  Any sufficiently advanced technology is indistinguishable from magic.


signature.asc
Description: PGP signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Dave F via talk]

You don't say what you want to output to but I use the command line 
program ogr2ogr (you will need to install all GDAL, I believe - 
https://gdal.org/index.html).


Example to convert to CSV:

ogr2ogr.exe -f csv output.csv input.geojson -lco GEOMETRY=AS_XY

You may have to adjust the GEOMETRY settings to get it to format to your 
desire.


It's also very useful for converting between co-ordinate systems eg 
(-s_srs EPSG:27700 -t_srs EPSG:4326")


DaveF

On 28/02/2024 15:16, Martin Trautmann via talk wrote:

Is there any JSON converter that you would recommend?

There are plenty of web services, but none of those worked for me, on

(70 MB)

There are plenty of starter tutorials, in order to use python,
javascript etc.

But those did not work either.

I tried dasel, without success (dasel version v2.5.0)

So I wonder what the best solution might be, apart of writing my own
converter (probably in perl).

What I would expect is hopefully a 100 % conversion - or an error log
which data could not be processed.

The problem here seems to be the nesting of several levels.

Thanks,
Martin


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Martin Trautmann via talk]

On 28.02.24 17:53, Colin Smale wrote:

> Have you looked at "jq"?

I had. I got the feeling that I would have to understand the full josm
structure first, while other approaches would learn about the data
structure and delimiters on their own.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Colin Smale]

> On 28/02/2024 17:30 CET Martin Trautmann via talk  
> wrote:
> 
>  
> On 28.02.24 16:43, Mike Thompson wrote:
> > Hi Martin,
> > 
> > Could you provide some more detail on what specifically you are
> > attempting to achieve? Converting a geojson file of points to CSV is
> > pretty easy, but once you get to linestrings, multi-linestrings,
> > polygons, etc. it gets difficult because in those cases the geometry
> > objects have a variable number of components.  
> 
> Hi Mike,
> 
> what I need is
> 
> Strassenna;HsNr_Zus;StrSchlues
> Stinnesstr.;8;02968
> 
> ..which I could grep easily, but I would also need to know in which
> "Gemeinde" that is, including their "Amtlicher Gemeindeschlüssel (AGS)"

Have you looked at "jq"?
Some tips here:
https://qmacro.org/blog/posts/2022/05/19/json-object-values-into-csv-with-jq/
https://richrose.dev/posts/linux/jq/jq-json2csv/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Martin Trautmann via talk]

On 28.02.24 16:43, Mike Thompson wrote:
> Hi Martin,
> 
> Could you provide some more detail on what specifically you are
> attempting to achieve? Converting a geojson file of points to CSV is
> pretty easy, but once you get to linestrings, multi-linestrings,
> polygons, etc. it gets difficult because in those cases the geometry
> objects have a variable number of components.  

Hi Mike,

what I need is

Strassenna;HsNr_Zus;StrSchlues
Stinnesstr.;8;02968

..which I could grep easily, but I would also need to know in which
"Gemeinde" that is, including their "Amtlicher Gemeindeschlüssel (AGS)"


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] recommendation for JSON to CSV converter

[Mike Thompson]

Hi Martin,

Could you provide some more detail on what specifically you are attempting
to achieve? Converting a geojson file of points to CSV is pretty easy, but
once you get to linestrings, multi-linestrings, polygons, etc. it gets
difficult because in those cases the geometry objects have a variable
number of components.

Mike

On Wed, Feb 28, 2024 at 8:26 AM Martin Trautmann via talk <
talk@openstreetmap.org> wrote:

> Is there any JSON converter that you would recommend?
>
> There are plenty of web services, but none of those worked for me, on
> <
> https://opendata.essen.de/sites/default/files/Hauskoordinaten_0822.geojson
> >
> (70 MB)
>
> There are plenty of starter tutorials, in order to use python,
> javascript etc.
>
> But those did not work either.
>
> I tried dasel, without success (dasel version v2.5.0)
>
> So I wonder what the best solution might be, apart of writing my own
> converter (probably in perl).
>
> What I would expect is hopefully a 100 % conversion - or an error log
> which data could not be processed.
>
> The problem here seems to be the nesting of several levels.
>
> Thanks,
> Martin
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk