Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-12-16 Thread Lars Francke
> Have you been considering how to handle the history of old anonymous edits? > This new history data should not reveal those user names but keep them > anonymous. User IDs and Usernames for those anonymous edits are left out of the dump. So the elements may not have an uid or user attribute. Th

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-12-16 Thread Jukka Rahkonen
Hi, Have you been considering how to handle the history of old anonymous edits? This new history data should not reveal those user names but keep them anonymous. -Jukka Rahkonen- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Lars Francke
>>> There is a dump, but it's a mysql dump so not easily readable. There may >>> be >>> a planet around somewhere as well but I don't think it will be >>> synchronised >>> to the actual shutdown time or have any history. >> >> If you'd be willing to share the mysql dump (I of course wouldn't need >

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Lars Francke
A quick status update and a link to the code. - I decided to dump num_changes too - One thing that startled me: planet.c converts _all_ relation member roles to lower case before dumping them. I'd consider this a bug but I'm sure there is a reason for this. Considering that neither the API nor Os

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Tom Hughes
On 12/11/09 17:37, Lars Francke wrote: >> There is a dump, but it's a mysql dump so not easily readable. There may be >> a planet around somewhere as well but I don't think it will be synchronised >> to the actual shutdown time or have any history. > > If you'd be willing to share the mysql dump (

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Matt Amos
On Thu, Nov 12, 2009 at 4:28 PM, Lars Francke wrote: > - As of now the XML is not indented. I use Woodstox[1] for XML output > and that doesn't have an option to "pretty print" the output. It is > not a problem for me but if it is requested I can use StaxMate or > something else to properly indent

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Lars Francke
>> I am partly done with my Java version. There are a few >> questions/problems/remarks: > > Is Java really up to this job from a performance point of view? I haven't done any performance comparisons between planet.c and my program but I believe that there won't be much of a difference. I could "r

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Tom Hughes
On 12/11/09 16:28, Lars Francke wrote: > I am partly done with my Java version. There are a few > questions/problems/remarks: Is Java really up to this job from a performance point of view? > - Is there a dump of the database available from just prior to the > switch from API 0.4 to 0.5? I could

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-12 Thread Lars Francke
> just remember that new code = new bugs ;-) hehe yeah that's true but I assure you: It would probably be worse if I tried to do it in C or as an Osmosis task. And as I said before: No harm in me trying. It will be your decision if its up to your standards. And I learned a great deal from Osmosis

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Andy Robinson (blackadder-lists)
Lars Francke [mailto:lars.fran...@gmail.com] >Sent: 11 November 2009 1:16 PM >To: Andy Robinson (blackadder-lists) >Cc: OpenStreetMap Dev >Subject: Re: [OSM-dev] Complete history of OSM data - questions and >discussion > >Andy, > >> Ultimately the formation of a m

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Lars Francke
>> I had not thought of that. When I first used OSM, segments were long >> gone so I tend to forget those. There are a few planet dumps from >> those time but none before 060403 and no diffs (which would be >> required to fully reconstruct the history). > > No, diffs only show the changes between t

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Matt Amos
On Wed, Nov 11, 2009 at 1:29 PM, Lars Francke wrote: > I had > brief discussions with Brett about Osmosis and incorporating certain > changes into it so I've spent quite some time in its source code. > Having said that: I probably won't program this as a new task for > Osmosis but as a standalone

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Lars Francke
>> - Am I correct in assuming that there are no general objections from >> the OSM server folks against such a dump? (Which would render the rest >> of this E-Mail useless ;-) > > the response has always been "if someone writes it, and it's good, > we'll run it" :-) That's all I wanted to hear :)

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Frederik Ramm
Hi, Lars Francke wrote: > I had not thought of that. When I first used OSM, segments were long > gone so I tend to forget those. There are a few planet dumps from > those time but none before 060403 and no diffs (which would be > required to fully reconstruct the history). No, diffs only show th

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Lars Francke
Andy, > Ultimately the formation of a mini project is probably needed. Input from > those like yourself willing to work on it and the will and time from others > who would need to support the work, including sysadmins. I had hoped that this could be (re-)solved rather unbureaucratic and without t

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Lars Francke
>> I understand that a lot of this data is >> available throughout the web using old snapshots and diffs but this >> comes in outdated formats and is by no way complete or easy to use. > > Keep in mind that while a full database dump will give you some things that > are not in the old planet files,

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Brett Henderson
On Wed, Nov 11, 2009 at 10:07 PM, Matt Amos wrote: > > > note that brett has been working on option (2) by using osmosis to > dump very historical diffs going back to the inception of the > database. you can see the experimental results in > http://planet.openstreetmap.org/history/ > I'm happy t

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Frederik Ramm
Hi, Lars Francke wrote: > I understand that a lot of this data is > available throughout the web using old snapshots and diffs but this > comes in outdated formats and is by no way complete or easy to use. Keep in mind that while a full database dump will give you some things that are not in th

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Matt Amos
On Wed, Nov 11, 2009 at 6:41 AM, Lars Francke wrote: > There are a few questions that probably need answering first and I > hope we can start a discussion about this: > - Am I correct in assuming that there are no general objections from > the OSM server folks against such a dump? (Which would ren

Re: [OSM-dev] Complete history of OSM data - questions and discussion

2009-11-11 Thread Andy Robinson (blackadder-lists)
ap.org] >On Behalf Of Lars Francke >Sent: 11 November 2009 6:42 AM >To: OpenStreetMap Dev >Subject: [OSM-dev] Complete history of OSM data - questions and discussion > >Hi! > >I and many (okay at least a few) others have shown interest in the >complete history data of OSM. I

[OSM-dev] Complete history of OSM data - questions and discussion

2009-11-10 Thread Lars Francke
Hi! I and many (okay at least a few) others have shown interest in the complete history data of OSM. I understand that a lot of this data is available throughout the web using old snapshots and diffs but this comes in outdated formats and is by no way complete or easy to use. I also had a look at