Re: [osmosis-dev] Problem with pbf import into a postgres DB
Hi Thomas, I think you have the wrong schema installed. There is no node_tags table in the schema. There used to be in older versions of the schema. Can you try a couple of things please? - Remove the validateSchemaVersion="no" option from your Osmosis command line. You don't need it and it can hide the real problem. - Re-create your schema from the 0.38 distribution. The 0.38 script doesn't create a node_tags table, it creates a hstore tags column on the nodes table. The below script should be the one you are using. http://svn.openstreetmap.org/applications/utils/osmosis/tags/0.38/package/script/pgsql_simple_schema_0.6.sql - Make sure that no errors occur during execution of the above script. - You may need to install "hstore" support in your database. Your PostgreSQL distribution should provide this script. - Note that the scripts are in the "script" directory, not in the "script/contrib" directory. Do NOT use the "script/contrib/apidb_0.6.sql" script. There are no step by step instructions on debugging Osmosis, however there are some basic instructions on configuring Eclipse. http://wiki.openstreetmap.org/wiki/Osmosis/Development#Eclipse_Setup If you do want to test it in Eclipse, run the existing JUnit tests in the "pgsnapshot" project from within Eclipse. Cheers, Brett On Thu, Dec 16, 2010 at 8:05 PM, Thomas Schreiner wrote: > Hello Brett, > > my english is also not so well, but we can at least communicate. > What I did is, I set up a postgres 8.4 database, create a user and a > database. Run the sql script pgsql_simple_schema_0.6.sql from the 0.38 > distribution out of the directory script/contrib. > > I downloaded a pbf file from download.geofabrik.de an try to import these > pbf file. > The first error message I got was about the schema version, so I turned the > validation of the schema of. > > Then I got the error with the index, no idea why the index is not created > in the sql script. Never then less I changed the code as described bevor. > > Now I get the error message that the node table has no tags column. That's > right, because there is another table called node_tags. A node can have > several tags, e.g. house number, telephone number, name, amenity, ... > > So I had also a look into the sql script which creates the postgres schema, > but I find nothing strange. > > Can you tell me how to set up osmosis via svn in eclipse to be able to > debug the application? Is there a step by step documentation? > > Thanks > > Thomas > > Original-Nachricht ---- > > Datum: Thu, 16 Dec 2010 18:58:51 +1100 > > Von: Brett Henderson > > An: Thomas Schreiner > > CC: "osmosis-dev@openstreetmap.org" > > Betreff: Re: [osmosis-dev] Problem with pbf import into a postgres DB > > > Hi Thomas, > > > > My German isn't great, but I'm guessing that the following message is > > complaining about a missing tags column in the nodes table. > > > > FEHLER: Spalte »tags« von Relation »nodes« existiert nicht > > > > I notice you're specifying the following option. > > validateSchemaVersion="no" > > > > Why are you doing that? You need to use a schema version that matches > your > > version of Osmosis. > > > > Brett > > > > > > On 16/12/2010, at 4:10, Thomas Schreiner > wrote: > > > > nicht > > -- > Thomas Schreiner > Grasslweg 4 > 85049 Ingolstadt > > GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt auch mit > gratis Notebook-Flat! http://portal.gmx.net/de/go/dsl > ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem with pbf import into a postgres DB
Hello Brett, my english is also not so well, but we can at least communicate. What I did is, I set up a postgres 8.4 database, create a user and a database. Run the sql script pgsql_simple_schema_0.6.sql from the 0.38 distribution out of the directory script/contrib. I downloaded a pbf file from download.geofabrik.de an try to import these pbf file. The first error message I got was about the schema version, so I turned the validation of the schema of. Then I got the error with the index, no idea why the index is not created in the sql script. Never then less I changed the code as described bevor. Now I get the error message that the node table has no tags column. That's right, because there is another table called node_tags. A node can have several tags, e.g. house number, telephone number, name, amenity, ... So I had also a look into the sql script which creates the postgres schema, but I find nothing strange. Can you tell me how to set up osmosis via svn in eclipse to be able to debug the application? Is there a step by step documentation? Thanks Thomas Original-Nachricht > Datum: Thu, 16 Dec 2010 18:58:51 +1100 > Von: Brett Henderson > An: Thomas Schreiner > CC: "osmosis-dev@openstreetmap.org" > Betreff: Re: [osmosis-dev] Problem with pbf import into a postgres DB > Hi Thomas, > > My German isn't great, but I'm guessing that the following message is > complaining about a missing tags column in the nodes table. > > FEHLER: Spalte »tags« von Relation »nodes« existiert nicht > > I notice you're specifying the following option. > validateSchemaVersion="no" > > Why are you doing that? You need to use a schema version that matches your > version of Osmosis. > > Brett > > > On 16/12/2010, at 4:10, Thomas Schreiner wrote: > > nicht -- Thomas Schreiner Grasslweg 4 85049 Ingolstadt GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt auch mit gratis Notebook-Flat! http://portal.gmx.net/de/go/dsl ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem with pbf import into a postgres DB
Hi Thomas, My German isn't great, but I'm guessing that the following message is complaining about a missing tags column in the nodes table. FEHLER: Spalte »tags« von Relation »nodes« existiert nicht I notice you're specifying the following option. validateSchemaVersion="no" Why are you doing that? You need to use a schema version that matches your version of Osmosis. Brett On 16/12/2010, at 4:10, Thomas Schreiner wrote: nicht ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem with pbf import into a postgres DB
her.Launcher.main(Launcher.java:352) at org.codehaus.classworlds.Launcher.main(Launcher.java:31) tho...@thomas-laptop:~/workspace/osmosis-SNAPSHOT-r24762M/bin$ I have no idea what's going wrong here, maybe I have to debug osmosis, but to set up the right configuration for debugging will take a while. Any hints are welcome. Thomas ---- Original-Nachricht > Datum: Wed, 15 Dec 2010 11:06:11 +0100 > Von: "Thomas Schreiner" > An: osmosis-dev@openstreetmap.org > Betreff: [osmosis-dev] Problem with pbf import into a postgres DB > Hello, > > I've downloaded the osmosis version 0.38 (latest). > I try to import a pbf file downloaded from download.geofabrik.de. > > I will always get an exception, that an index can not be dropped. Why. > See the details below. > > tho...@thomas-laptop:~/workspace/osmosis-0.38/bin$ ./osmosis --rb > file="/Massenspeicher/bayern.osm.pbf" --wp user="osm" database="osmdb" > password="xxx" validateSchemaVersion="no" nodeLocationStoreType="TempFile" > 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run > INFO: Osmosis Version 0.38 > 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run > INFO: Preparing pipeline. > 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run > INFO: Launching pipeline execution. > 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run > INFO: Pipeline executing, waiting for completion. > 15.12.2010 01:45:34 > org.springframework.beans.factory.xml.XmlBeanDefinitionReader > loadBeanDefinitions > INFO: Loading XML bean definitions from class path resource > [org/springframework/jdbc/support/sql-error-codes.xml] > 15.12.2010 01:45:35 org.springframework.jdbc.support.SQLErrorCodesFactory > > INFO: SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, > MySQL, Oracle, PostgreSQL, Sybase] > 15.12.2010 01:45:35 > org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager > waitForCompletion > SCHWERWIEGEND: Thread for task 1-rb failed > org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad > SQL grammar [DROP INDEX idx_relation_members_member_id_and_type]; nested > exception is org.postgresql.util.PSQLException: FEHLER: Index > »idx_relation_members_member_id_and_type« existiert nicht > at > org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:98) > at > org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) > at > org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) > at > org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) > at > org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:406) > at > org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:518) > at > org.springframework.jdbc.core.simple.SimpleJdbcTemplate.update(SimpleJdbcTemplate.java:248) > at > org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.IndexManager.prepareForLoad(IndexManager.java:103) > at > org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.CopyFilesetLoader.run(CopyFilesetLoader.java:92) > at > org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlCopyWriter.complete(PostgreSqlCopyWriter.java:108) > at > crosby.binary.osmosis.OsmosisBinaryParser.complete(OsmosisBinaryParser.java:33) > at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:17) > at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:36) > at java.lang.Thread.run(Thread.java:662) > Caused by: org.postgresql.util.PSQLException: FEHLER: Index > »idx_relation_members_member_id_and_type« existiert nicht > at > org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062) > at > org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795) > at > org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:299) > at > org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:228) > at > org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:228) >
[osmosis-dev] Problem with pbf import into a postgres DB
Hello, I've downloaded the osmosis version 0.38 (latest). I try to import a pbf file downloaded from download.geofabrik.de. I will always get an exception, that an index can not be dropped. Why. See the details below. tho...@thomas-laptop:~/workspace/osmosis-0.38/bin$ ./osmosis --rb file="/Massenspeicher/bayern.osm.pbf" --wp user="osm" database="osmdb" password="xxx" validateSchemaVersion="no" nodeLocationStoreType="TempFile" 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.38 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run INFO: Preparing pipeline. 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline execution. 15.12.2010 00:59:04 org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing, waiting for completion. 15.12.2010 01:45:34 org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions INFO: Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml] 15.12.2010 01:45:35 org.springframework.jdbc.support.SQLErrorCodesFactory INFO: SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase] 15.12.2010 01:45:35 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion SCHWERWIEGEND: Thread for task 1-rb failed org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [DROP INDEX idx_relation_members_member_id_and_type]; nested exception is org.postgresql.util.PSQLException: FEHLER: Index »idx_relation_members_member_id_and_type« existiert nicht at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:98) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:406) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:518) at org.springframework.jdbc.core.simple.SimpleJdbcTemplate.update(SimpleJdbcTemplate.java:248) at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.IndexManager.prepareForLoad(IndexManager.java:103) at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.CopyFilesetLoader.run(CopyFilesetLoader.java:92) at org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlCopyWriter.complete(PostgreSqlCopyWriter.java:108) at crosby.binary.osmosis.OsmosisBinaryParser.complete(OsmosisBinaryParser.java:33) at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:17) at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:36) at java.lang.Thread.run(Thread.java:662) Caused by: org.postgresql.util.PSQLException: FEHLER: Index »idx_relation_members_member_id_and_type« existiert nicht at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:299) at org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:228) at org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:228) at org.springframework.jdbc.core.JdbcTemplate$1UpdateStatementCallback.doInStatement(JdbcTemplate.java:508) at org.springframework.jdbc.core.JdbcTemplate$1UpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:395) ... 9 more 15.12.2010 01:45:35 org.openstreetmap.osmosis.core.Osmosis main SCHWERWIEGEND: Execution aborted. org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed. at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146) at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92) at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invok
