Re: [osmosis-dev] Database is empty after five days of importing
It's not currently possible. The --truncate-apidb and --write-apidb tasks have to be run via separate Osmosis invocations (to ensure one completes before the other starts), and cannot be made to run inside a single transaction. Note that the import itself does run in a transaction, but doesn't include the truncate. On Fri, Mar 26, 2010 at 1:26 AM, Klemens toural...@gmail.com wrote: Thanks. Any hints on how this can be done with osmosis? On 24.03.2010 21:12, Peter Körner wrote: Import in a transaction. ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Database is empty after five days of importing
Thanks. Any hints on how this can be done with osmosis? On 24.03.2010 21:12, Peter Körner wrote: Import in a transaction. ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Database is empty after five days of importing
You're right, it's no fun at all. Well here's what I did to try it with a smaller country excerpt (102 MB compressed) this night: nohup bzcat ~/install/osm-dumps/austria.osm.bz2 | ./osmosis-0.34/bin/osmosis --read-xml-0.6 file=- --write-apidb-0.6 populateCurrentTables=yes host=localhost database=... user=... password=... validateSchemaVersion=no ~/nohup.out Still all the tables are empty. It really looks like a roleback or something, e.g. the nodes table has inserted tuples but alle the current_xxx tables don't have. The problem is that I haven't got logs - nothing is written to nohup.out. I could see only the few statements at the beginning of the import (pipeline executing, waiting for completion). I'm trying with an even smaller dataset right now. What is actually the right way if I first want to import for example one country from http://download.geofabrik.de/osm/europe/ and then, later, whole Europe into the same DB? I only need a read-only DB. Is it possible (I hope so) and if yes, which paramters if osmosis do I need to set? 2010/3/24 Brett Henderson br...@bretth.com: Have you tried it with a smaller dataset first to make sure your import process works? It's not much fun waiting for several days only to find that something trivial blocks successful completion. Without knowing what you've done, it sounds like you encountered a failure and the database rolled back. Did Osmosis complete successfully? Do you have a complete log of its output? ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Database is empty after five days of importing
Hi, Klemens wrote: So at some time in the future when I want to import Europe into a DB that contains Austria I will have to create a delta .osm first using osmosis? Possible, but impractical, because you would spend more time to create the delta than you save by re-using the existing data. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Database is empty after five days of importing
Am 24.03.2010 20:10, schrieb Klemens: That makes sense. My requirement is that I must continue querying the old data while importing the new, larger data - which can take a long time as we know. How can I achieve this? Import in a transaction. Peter ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Database is empty after five days of importing
Hi, Klemens wrote: That makes sense. My requirement is that I must continue querying the old data while importing the new, larger data - which can take a long time as we know. How can I achieve this? I would suggest to create a second database instance (same server process, different database name, same table names), import into that, then drop the old database and rename the newly imported. This will give you a short service interruption. If you cannot tolerate any service interruption, you can use the pgbouncer proxy which can be reconfigured to connect to another database while still serving active connections to the old database. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Database is empty after five days of importing
Hi, I made an import of the planet.osm file using osmosis - the process ran for about five days. Now it finished but effectively the postgresql database is empty. In the statistics I can see that Tuples inserted for the nodes table is 564,050,700 and the size is 117 GB. But there are no rows in it. select * from nodes; returns 0 rows. Can anybody tell me what happened? Thanks, Klemens ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
