> In the past I could generally get the query to complete by running it once, > cancelling it, then running it a second time (maybe the 2nd time more data > was cached). Now I get either infinite timeout or a server rejection. > > The query is: > > <osm-script timeout="990" element-limit="1073741824"> > <query type="node"> > <has-kv k="amenity" v="drinking_water"/> > </query> > <print/> > </osm-script>
I'm glad that you bring up the whole subject. It's a challenging query for Overpass API. Malcolm has already sketched the long-term solution: operating from a SSD will largely solve the performance limitations, but currently I'm not aware of any cost-effective hosting options with large SSDs (more than 300 GB). Please do not use a cancel-restart strategy. This may double the load because not in all cases the Apache server cancels the abandoned query. It is possible that this strategy causes the rejection because the server rejects concurrent requests from the same IP. In fact, if a query is already running a second query from the same IP then the new request is first postponed for up to 15 seconds. It is immediately started if the former request completes during this time. If the former request doesn't complete within 15 seconds then the newer request is finally rejected. I suggest the following strategy: Please prefer the Rambler instance. It is for this use case probably the fastest. Then, feel free to use a much larger timeout. On a test run, the query had taken 12 minutes, so with a safety margin factor, please try 3600 seconds instead of 990. On the other hand, please omit the element-limit (and thus use the default of 512M instead). That's much more than necessary and the server is rather picky on RAM consuming requests. If the request is still rejected, please retry every few minutes. Peaks most often only last for less than an hour but aren't that regular. And trying a request doesn't hurt anybody. Cheers, Roland _______________________________________________ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk