Re: [osmosis-dev] Has the behaviour of Osmosis changed regarding deletions in .osc?
On Thu, Nov 12, 2009 at 11:23 AM, Frederik Ramm wrote: > Hi, > > > Brett Henderson wrote: > >> I always thought that osmosis would only write the object "header" >>(the >deleted, and omit the tags and members. Now I see that it in fact >>includes the full object. Has this behaviour changed some time, or has >>this always been the case? >> >> >> Nothing should have changed in that space. The Osmosis --write-xml-change >> task will write everything that is passed to it in the pipeline. So if the >> original change derivation task provides tags and members they'll be written >> to the output file. >> > > Oh right, then that was my misunderstanding. I remembered from looking at > the planet .osc files, and then tried it out by comparing two .osm files and > thought the behaviour must have changed. > > Interestingly, the .osc files on planet seem to contain the full set of > tags for deleted *nodes*, but no tags for deleted ways or relations. That is strange. Do you have an example of each? It would be interesting to see what the API returns, and beyond that what is actually in the db. Brett ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Relations with > 65k members
On Thu, Nov 12, 2009 at 9:36 AM, Frederik Ramm wrote: > Hi, > >we just had a problem with diff updates because someone created > relations with more than 65k members and Osmosis tries to store them as > a char which raised an exception. > > I did a quick hack that would simply drop the excessive members with a > warning: > > http://www.remote.org/frederik/tmp/remove_excessive_members.diff > > I'm not checking that in. > > I thought about simply patching Osmosis to use an integer but was > informed by Jon Burgess that osm2pgsql (which is next in my toolchain) > would carp on > 65k members as well so I didn't bother - especially as > it turned out that the relations in question were an error in somebody's > automated import and have been meanwhile fixed by Lennard (they > contained the same element over and over). > If > 65k members is legitimate then feel free to change Osmosis. I only picked smaller data types to save some memory when doing things like large sorts. If the API allows it then perhaps Osmosis should too. Another one that will have to be changed eventually is entity ids. They'll only allow up to 2^31 currently (signed integer) which will probably break eventually given the rapid growth of the database. ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Has the behaviour of Osmosis changed regarding deletions in .osc?
Hi, Brett Henderson wrote: >I always thought that osmosis would only write the object "header" > (the deleted, and omit the tags and members. Now I see that it in fact > includes the full object. Has this behaviour changed some time, or has > this always been the case? > > > Nothing should have changed in that space. The Osmosis > --write-xml-change task will write everything that is passed to it in > the pipeline. So if the original change derivation task provides tags > and members they'll be written to the output file. Oh right, then that was my misunderstanding. I remembered from looking at the planet .osc files, and then tried it out by comparing two .osm files and thought the behaviour must have changed. Interestingly, the .osc files on planet seem to contain the full set of tags for deleted *nodes*, but no tags for deleted ways or relations. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Relations with > 65k members
Hi, Frederik Ramm wrote: > I thought about simply patching Osmosis to use an integer but was > informed by Jon Burgess that osm2pgsql (which is next in my toolchain) > would carp on > 65k members as well Turns out osm2pgsql in its default config allows only up to 32767 members. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Relations with > 65k members
Hi, we just had a problem with diff updates because someone created relations with more than 65k members and Osmosis tries to store them as a char which raised an exception. I did a quick hack that would simply drop the excessive members with a warning: http://www.remote.org/frederik/tmp/remove_excessive_members.diff I'm not checking that in. I thought about simply patching Osmosis to use an integer but was informed by Jon Burgess that osm2pgsql (which is next in my toolchain) would carp on > 65k members as well so I didn't bother - especially as it turned out that the relations in question were an error in somebody's automated import and have been meanwhile fixed by Lennard (they contained the same element over and over). Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Has the behaviour of Osmosis changed regarding deletions in .osc?
On Thu, Nov 12, 2009 at 6:53 AM, Frederik Ramm wrote: > Hi, > >I always thought that osmosis would only write the object "header" > (the deleted, and omit the tags and members. Now I see that it in fact > includes the full object. Has this behaviour changed some time, or has > this always been the case? > Nothing should have changed in that space. The Osmosis --write-xml-change task will write everything that is passed to it in the pipeline. So if the original change derivation task provides tags and members they'll be written to the output file. I don't believe the database contains tags and members for deleted items so changesets on the planet server shouldn't contain them either. But if you're deriving a change between two osm files, then it might be possible for the tags and members to be included. In what scenario (ie. which task is producing the change) is the tag and member info being included? Brett ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Has the behaviour of Osmosis changed regarding deletions in .osc?
Hi, I always thought that osmosis would only write the object "header" (the http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Upload-Task
No problem. It's quite trivial code. Marcus On Wed, 11 Nov 2009 23:14:40 +1100, Brett Henderson wrote: > Sure, sounds great! If you're prepared to help support any issues with it > then check it in. > > On Wed, Nov 11, 2009 at 7:33 PM, wrote: > >> >> >> Hello Brett, >> >> I have the Java-code required to open a changeset and upload an OSC-file. >> Are you interested in an "upload-change"-task for Osmosis? >> It can do all kinds of things with changes but currently python >> is required to upload them. So this is the one feature missing >> for a consistent set of tasks. ;) >> >> >> Marcus >> ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Upload-Task
Sure, sounds great! If you're prepared to help support any issues with it then check it in. On Wed, Nov 11, 2009 at 7:33 PM, wrote: > > > Hello Brett, > > I have the Java-code required to open a changeset and upload an OSC-file. > Are you interested in an "upload-change"-task for Osmosis? > It can do all kinds of things with changes but currently python > is required to upload them. So this is the one feature missing > for a consistent set of tasks. ;) > > > Marcus > ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem using Osmosis (PluginLifecycleException)
On Wed, Nov 11, 2009 at 10:52 PM, wrote:
> On Wed, 11 Nov 2009 22:37:27 +1100, Brett Henderson
> wrote:
> >> http://ant.apache.org/manual/CoreTasks/jar.html
> >> That would be the sub-element.
> >>
> >
> > Unless I'm misunderstanding the attribute element that's not quite what I
> > meant.
> >
> > I already have the following snippet in build.xml.
> >
> > > value="org.openstreetmap.osmosis.core.Osmosis"/>
> >
> >
> > > value="${project.version} (${TODAY})"/>
> >
> >
> >
> > I could add a classpath element, but I don't know how to add it
> dynamically
> > so that I don't have to hard code the list of jars.
>
>
> I haven't tried it but this could do the trick:
>
>
> http://stackoverflow.com/questions/1456852/ant-commands-to-print-fileset-into-a-file-one-match-per-line
>
>
>
>
>
>
> value="org.openstreetmap.osmosis.core.Osmosis"/>
>
>
> value="${project.version} (${TODAY})"/>
>
>
>
>
> Now how to change ";" into ":" in ${jars}
> But maybe it works anyway.
>
Okay, cool. I'll try it out when I get a chance.
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem using Osmosis (PluginLifecycleException)
On Wed, 11 Nov 2009 22:37:27 +1100, Brett Henderson
wrote:
>> http://ant.apache.org/manual/CoreTasks/jar.html
>> That would be the sub-element.
>>
>
> Unless I'm misunderstanding the attribute element that's not quite what I
> meant.
>
> I already have the following snippet in build.xml.
>
> value="org.openstreetmap.osmosis.core.Osmosis"/>
>
>
> value="${project.version} (${TODAY})"/>
>
>
>
> I could add a classpath element, but I don't know how to add it
dynamically
> so that I don't have to hard code the list of jars.
I haven't tried it but this could do the trick:
http://stackoverflow.com/questions/1456852/ant-commands-to-print-fileset-into-a-file-one-match-per-line
Now how to change ";" into ":" in ${jars}
But maybe it works anyway.
Marcus
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem using Osmosis (PluginLifecycleException)
On Wed, Nov 11, 2009 at 4:03 PM, Marcus Wolschon < marcus.wolsc...@googlemail.com> wrote: > On Tue, Nov 10, 2009 at 11:03 PM, Brett Henderson > wrote: > > >>> Changing osmosis.bat is not the problem but how can I do an entry > > Does > > anybody know how to build a classpath list based on the jars contained in > a > > directory within ant? > > http://ant.apache.org/manual/CoreTasks/jar.html > That would be the sub-element. > Unless I'm misunderstanding the attribute element that's not quite what I meant. I already have the following snippet in build.xml. I could add a classpath element, but I don't know how to add it dynamically so that I don't have to hard code the list of jars. ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Upload-Task
Hello Brett, I have the Java-code required to open a changeset and upload an OSC-file. Are you interested in an "upload-change"-task for Osmosis? It can do all kinds of things with changes but currently python is required to upload them. So this is the one feature missing for a consistent set of tasks. ;) Marcus ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
