Hi Mike, good catch, I forgot to add the ...Factory.java file. Hopefully I haven't forgotten any more files.
Cheers On 22 May 2010 04:08, Mike N. <nice...@att.net> wrote: > Hi andrzej, > > I tried to apply the patch, but I get an undefined symbol when trying to > build 'FlattenFilterFactory'. Am I missing part of the patch or is there > another setting to create the Factory? > > Thanks, > > Mike Nice > > -------------------------------------------------- > From: "andrzej zaborowski" <balr...@gmail.com> > Sent: Friday, May 21, 2010 2:40 PM > To: "David Carmean" <d...@halibut.com> > Cc: <osmosis-dev@openstreetmap.org>; "OSM US Talk" > <talk...@openstreetmap.org> > Subject: Re: [Talk-us] Cloudmade california.osm: dups or osmosis bug? > >> Hi, >> >> On 21 May 2010 01:54, David Carmean <d...@halibut.com> wrote: >>> >>> Is it the dataset or osmosis that is giving me a single 2-node duplicate >>> for each postGIS table created by osmosis? >> >> Not sure if it's related, but I noticed that many of the CloudMade >> extracts can't be processed using osmosis (even though they're >> generated with osmosis), except for the operations that don't try to >> validate the order of elements. It seems they often contain >> consecutive elements with the same id, but possibly different version. >> I have a little patch (attached) to add a new filter (--ff) that >> "flattens" the file. It does the same operation on sorted entity >> streams as --simplify-change does on change streams, unfortunately >> "--simplify" is taken by a different operation now. I decided it was >> faster to modify osmosis than to download the planet and make my own >> un-broken extracts. >> >> Cheers >> (I'm not subscribed to osmosis-dev) >> > > > >> _______________________________________________ >> Talk-us mailing list >> talk...@openstreetmap.org >> http://lists.openstreetmap.org/listinfo/talk-us >> >
Index: src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java =================================================================== --- src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java (revision 0) +++ src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java (revision 0) @@ -0,0 +1,91 @@ +/* This software is released into the Public Domain. + * See copying.txt for details. */ +package org.openstreetmap.osmosis.core.filter.v0_6; + +import org.openstreetmap.osmosis.core.container.v0_6.BoundContainer; +import org.openstreetmap.osmosis.core.container.v0_6.EntityContainer; +import org.openstreetmap.osmosis.core.container.v0_6.EntityProcessor; +import org.openstreetmap.osmosis.core.domain.v0_6.Entity; +import org.openstreetmap.osmosis.core.task.v0_6.Sink; +import org.openstreetmap.osmosis.core.task.v0_6.SinkSource; + + +/** + * Flatten / simplify a sorted entity stream. + * (similar to --simplify-change) + */ +public class FlattenFilter implements SinkSource { + private Sink sink; + private EntityContainer previous_container; + + /** + * Creates a new instance. + */ + public FlattenFilter() { + } + + /** + * Process a node, way or relation. + * + * @param current_container + * The entity container to be processed. + */ + public void process(EntityContainer current_container) { + if (previous_container == null) { + previous_container = current_container; + return; + } + + Entity current = current_container.getEntity(); + Entity previous = previous_container.getEntity(); + + if (current.getId() != previous.getId() || + current.getClass() != previous.getClass()) { + sink.process(previous_container); + previous_container = current_container; + return; + } + + if (current.getVersion() > previous.getVersion()) + previous_container = current_container; + } + + /** + * Process the bound. + * + * @param boundContainer + * The bound to be processed. + */ + public void process(BoundContainer boundContainer) { + /* By default, pass it on unchanged */ + sink.process(boundContainer); + } + + /** + * {...@inheritdoc} + */ + public void complete() { + /* + * If we've stored entities temporarily, we now need to + * forward the stored ones to the output. + */ + if (previous_container != null) + sink.process(previous_container); + + sink.complete(); + } + + /** + * {...@inheritdoc} + */ + public void release() { + sink.release(); + } + + /** + * {...@inheritdoc} + */ + public void setSink(Sink sink) { + this.sink = sink; + } +} Index: src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java =================================================================== --- src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java (revision 0) +++ src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java (revision 0) @@ -0,0 +1,26 @@ +/* This software is released into the Public Domain. + * See copying.txt for details. */ +package org.openstreetmap.osmosis.core.filter.v0_6; + +import org.openstreetmap.osmosis.core.pipeline.common.TaskConfiguration; +import org.openstreetmap.osmosis.core.pipeline.common.TaskManager; +import org.openstreetmap.osmosis.core.pipeline.common.TaskManagerFactory; +import org.openstreetmap.osmosis.core.pipeline.v0_6.SinkSourceManager; + +/** + * The task manager factory for a flatten/simplify filter. + */ +public class FlattenFilterFactory extends TaskManagerFactory { + /** + * {...@inheritdoc} + */ + @Override + protected TaskManager createTaskManagerImpl( + TaskConfiguration taskConfig) { + return new SinkSourceManager( + taskConfig.getId(), + new FlattenFilter(), + taskConfig.getPipeArgs() + ); + } +} Index: src/org/openstreetmap/osmosis/core/TaskRegistrar.java =================================================================== --- src/org/openstreetmap/osmosis/core/TaskRegistrar.java (revision 21311) +++ src/org/openstreetmap/osmosis/core/TaskRegistrar.java (working copy) @@ -45,6 +45,7 @@ import org.openstreetmap.osmosis.core.filter.v0_6.UsedNodeFilterFactory; import org.openstreetmap.osmosis.core.filter.v0_6.WayKeyFilterFactory; import org.openstreetmap.osmosis.core.filter.v0_6.WayKeyValueFilterFactory; +import org.openstreetmap.osmosis.core.filter.v0_6.FlattenFilterFactory; import org.openstreetmap.osmosis.core.merge.v0_6.ChangeAppenderFactory; import org.openstreetmap.osmosis.core.merge.v0_6.ChangeMergerFactory; import org.openstreetmap.osmosis.core.merge.v0_6.ChangeSimplifierFactory; @@ -153,8 +154,10 @@ factoryRegister.register("read-xml", new XmlReaderFactory()); factoryRegister.register("fast-read-xml", new FastXmlReaderFactory()); factoryRegister.register("rx", new XmlReaderFactory()); - factoryRegister.register("read-xml-change", new XmlChangeReaderFactory()); - factoryRegister.register("upload-xml-change", new XmlChangeUploaderFactory()); + factoryRegister.register("flatten", new FlattenFilterFactory()); + factoryRegister.register("ff", new FlattenFilterFactory()); + factoryRegister.register("read-xml-change", new XmlChangeReaderFactory()); + factoryRegister.register("upload-xml-change", new XmlChangeUploaderFactory()); factoryRegister.register("rxc", new XmlChangeReaderFactory()); factoryRegister.register("sort", entitySorterFactory06); factoryRegister.register("s", entitySorterFactory06);
_______________________________________________ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev