On Tue, Jun 23, 2009 at 6:11 PM, Brett Henderson <br...@bretth.com> wrote:
> Karl Newman wrote: > >> >> What's happening there is that the node-key-value and way-key-value are >> ANDed together (which would leave you with only ways which match your tags >> and are composed of nodes tagged place=city), and you want an OR instead. >> You were sort of on the right track with the pipes, but what you need to do >> is use the "tee" function and apply the node-key-value filter to one leg of >> the tee, and apply the way-key-value filter to the other leg of the tee, >> then use the "merge" function to join the results. It would look something >> like this: >> >> ./osmosis-0.31/bin/osmosis --read-xml file="planet.bz2" --tee >> outputCount=2 outPipe.0="nodes" outPipe.1="ways" --node-key-value >> keyValueList="place.city" inPipe.0="nodes" --way-key-value >> keyValueList="highway.motorway,highway.motorway_link,highway.motorway_junction,highway.trunk,highway.trunk_link" >> inPipe.0="ways" --merge --write-xml file="basemap.osm" >> >> I'm not sure if that will work exactly as written. You may need to add >> outPipe arguments to the node-key-value and way-key-value filters and then >> reference them as inPipe.0 and inPipe.1 arguments to the merge task. >> > You may trigger a deadlock in this situation ... I've been waiting for > somebody to try this out for a long time :-) > > While it's possible to construct a pipeline that tees a single dataset into > multiple streams before merging them back together again, it is problematic > from thread synchronisation point of view because the same input thread is > feeding two inputs of another thread. Using a --buffer task within both > paths of the branch may help because it de-couples the threads somewhat with > a buffer. > > The --read-xml task creates a thread which passes data into the --tee task. > The --tee task doesn't create a thread, it just uses the existing thread to > pass incoming data to all consumers. The --node-key-value and > --way-key-value also use the existing thread to write to their destination > which in both cases is the --merge task. The --merge task creates a new > thread which reads the incoming data from both of its inputs, but both > inputs are coming from a single thread (ie. the original --read-xml thread). > The --merge thread may read from one input, then start waiting for a > specific value on the the other input and never receive it. > > But if it works let me know. I'm curious :-) > > Brett > Hmm... I didn't look at the code too closely. I thought the tee created separate threads. I'm trying to see what might cause a deadlock--it looks like it would happen in DataPostBox if anywhere--but it's not obvious what might trigger it. I guess what could happen is if the merge task is trying to get entities from both pipelines to compare them, and there isn't anything available in one of the pipelines, that might deadlock it. I guess someone will have to try it and see! Karl
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
