Re: [osmosis-dev] Error while applying updates to planet.osm

2010-06-23 Thread Frederik Ramm 
Tyler,

sounds like either your planet or your change file is corrupt. Check 
the files with "xmllint --stream". Re-download the one that is broken.

> Next, I ran the commands below to get my updates and simplify them:
> osmosis --rri workingDirectory=. --wxc update.osm.gz
> gzip -d -c update.osm.gz | /usr/local/osmosis/bin/osmosis
> --read-xml-change file=/dev/stdin --simplify-change --write-xml-change
> file=- | gzip -9 -c> update.unique.osm.gz
> 
> Finally, I attempted to apply these changes to my planet.osm file:
> bzcat planet-100618.osm.bz2 | /usr/local/osmosis/bin/osmosis
> --read-xml-change file="update.unique.osm.gz" --read-xml file=/dev/stdin
> --apply-change --write-xml file=- | bzip2 -c > planet.osm.bz2

All that can be shortened to this one line:

bzcat planet-100618.osm.bz2 | osmosis --rri --simc --rx - --ac --wx - | 
bzip2 > planet.osm.bz2

But it is probably good to do it in single steps while still testing the 
process.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev

[osmosis-dev] Error while applying updates to planet.osm

2010-06-23 Thread Tyler Gunn 

I recently downloaded the planet-100618.osm.bz2 file from one of the OSM
planet mirrors.
I also downloaded the state file with contents:
#Thu Jun 17 21:02:07 UTC 2010
sequenceNumber=5039
txnMaxQueried=0
timestamp=2010-06-17T21\:00\:00Z
txnReadyList=
txnMax=0
txnActiveList=

I configuring my repository ( see
http://wiki.openstreetmap.org/wiki/Using_planet.osm ).

Next, I ran the commands below to get my updates and simplify them:
osmosis --rri workingDirectory=. --wxc update.osm.gz
gzip -d -c update.osm.gz | /usr/local/osmosis/bin/osmosis
--read-xml-change file=/dev/stdin --simplify-change --write-xml-change
file=- | gzip -9 -c> update.unique.osm.gz

Finally, I attempted to apply these changes to my planet.osm file:
bzcat planet-100618.osm.bz2 | /usr/local/osmosis/bin/osmosis
--read-xml-change file="update.unique.osm.gz" --read-xml file=/dev/stdin
--apply-change --write-xml file=- | bzip2 -c > planet.osm.bz2

After what seemed like an eternity I got the following exception:

SEVERE: Thread for task 1-read-xml-change failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
has occurred, aborting.
at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:76)
at
org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:132)
at
org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:182)
at
org.openstreetmap.osmosis.core.merge.v0_6.impl.DataPostboxChangeSink.process(DataPostboxChangeSink.java:34)
at
org.openstreetmap.osmosis.core.merge.v0_6.impl.SortedDeltaChangePipeValidator.process(SortedDeltaChangePipeValidator.java:63)
at
org.openstreetmap.osmosis.core.xml.v0_6.impl.ChangeSourceElementProcessor$ChangeSinkAdapter.process(ChangeSourceElementProcessor.java:134)
at
org.openstreetmap.osmosis.core.xml.v0_6.impl.NodeElementProcessor.end(NodeElementProcessor.java:117)
at
org.openstreetmap.osmosis.core.xml.v0_6.impl.OsmChangeHandler.endElement(OsmChangeHandler.java:94)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown
Source)
at
org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown
Source)
at
org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:195)
at
org.openstreetmap.osmosis.core.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:107)
at java.lang.Thread.run(Thread.java:636)
23-Jun-2010 1:46:01 PM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion
SEVERE: Thread for task 2-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse
xml file /dev/stdin.  publicId=(null), systemId=(null),
lineNumber=116829631, columnNumber=20.
at
org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:113)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.xml.sax.SAXParseException: The value of attribute "id"
associated with an element type "node" must not contain the '<' character.
at
org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown
Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown
Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown
Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown
Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown
Source)
at org.apache.xerces.impl.XMLScanner.scanAttributeValue(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanAttribute(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)

Re: [osmosis-dev] [OSM-dev] Problem in rake db:migrate with postgre

2010-06-23 Thread Brett Henderson 
Hi Shaun,

I haven't had a chance to get a working rails installation running.  I need
an updated schema to make the Osmosis changes.  Are you able to run the
following command against a newly created database and provide me with the
resultant SQL file?

pg_dump -O -v -f "apidb_0.6.sql" api06

In the above command, replace the final api06 with the name of your
database.

Brett

On Sun, Jun 13, 2010 at 9:48 AM, Shaun McDonald
wrote:

> Hi Brett,
>
>
> http://git.openstreetmap.org/?p=rails.git;a=blob;f=db/migrate/051_add_status_to_user.rb;h=cc8a2f23848af8ffaf86dc6f5bb65cb42597d2d3;hb=180228b7e36224fefd5540306d7fb77799d4b827
>
> It's now a status column rather than active and visible.
>
> Shaun
>
> On 12 Jun 2010, at 17:30, Brett Henderson wrote:
>
> The Osmosis error message seems to indicate that the "active" column on the
> users table no longer exists.  I haven't checked the recent migrations to
> figure out exactly what has changed.
>
> Currently when populating a datatabase, Osmosis creates any required users
> and makes them active (active = 1 I think).  If active no longer exists, the
> replacement column will need to be populated with an equivalent value.
>
> Brett
>
> On Fri, Jun 11, 2010 at 6:43 PM, M Naveed Akram  wrote:
>
>> Brett: the issue is still there in the latest build of osmosis..
>> can u please guide a little if i want to resolve it
>>
>>
>> On Mon, Jun 7, 2010 at 12:44 PM, M Naveed Akram  wrote:
>>
>>> can anyone tell me that to which schema migration version number the new
>>> osmosis works fine
>>>
>>>
>>> On Sun, Jun 6, 2010 at 7:35 AM, Brett Henderson wrote:
>>>
 On Fri, Jun 4, 2010 at 10:38 PM, Ævar Arnfjörð Bjarmason <
 ava...@gmail.com> wrote:

> On Fri, Jun 4, 2010 at 09:40, M Naveed Akram  wrote:
> > thanks to all
> > UTF8 issue resolved.
> > but now its osmosis
>
> Last I checked osmosis hadn't been fixed to work with the latest
> schema changes in the rails port, I run the Debian osmosis version,
> but it wasn't fixed in trunk either (at least the newest user group
> thing).
>
> Maybe that's what's going on, I just use an older version of the rails
> port for my importing purposes.
>
> Due to the way osmosis does imports it's fairly susceptible to
> breaking like this due to relatively small schema changes.
>

 I wasn't aware of the breakage until now.  The database I use for unit
 tests is ancient.  I won't get to this for a while, but if anybody else
 wants to take a look I'd appreciate it muchly.

 Brett


>>>
>>>
>>> --
>>> Regards
>>> M Naveed Akram
>>> http://www.google.com/profiles/cmnajs
>>>
>>>
>>
>>
>> --
>> Regards
>> M Naveed Akram
>> http://www.google.com/profiles/cmnajs
>>
>>
> ___
> osmosis-dev mailing list
> osmosis-dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/osmosis-dev
>
>
>
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev

Re: [osmosis-dev] error caused by Caused by: org.postgresql.util.PSQLException: ERROR: column "active" of relation "users" does not exist

2010-06-23 Thread Brett Henderson 
Hi Michael,

There were some changes made to the schema recently that have broken
Osmosis.  Fixing Osmosis is on my list of things to do, but I'm not sure
when I'll get to do it.

Brett

On Wed, Jun 23, 2010 at 7:10 PM, Michael musset wrote:

> Hi,
>
> I've a france.osm from geofabrik
>
> i'm trying to import it into my database with osmosis with this command :
>
> ./osmosis --read-xml file="/home/mickael/planet/france.osm" --write-apidb
> populateCurrentTables=yes host="localhost" database="openstreetmap"
> user="openstreetmap" validateSchemaVersion=no
>
>
> the osmosis come from :
>
> wget 
> http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-bin-latest.tgz 
> 
> tar xvfz osmosis-bin-latest.tgz
>
> cd osmosis-0.35
> chmod a+x bin/osmosis
> bin/osmosis
>
>
> my railsport config is :
> test:
>   adapter: postgresql
>   database: osm_test
> #  username: osm_test
> #  password: osm_test
>   host: localhost
>   encoding: utf8
>
> production:
>   adapter: postgresql
>   database: osm
> #  username: osm
> #  password: osm
>   host: localhost
>   encoding: utf8
>
> development:
>   adapter: postgresql
>   database: openstreetmap
> #  username: openstreetmap
> #  password: openstreetmap
>   host: localhost
>   encoding: utf8
>
>
>
> when i've created the database i've done :
>
> createdb -E UTF8 -O openstreetmap openstreetmap
> createdb -E UTF8 -O openstreetmap osm_test
> createdb -E UTF8 -O openstreetmap osm
>
> and
>
> psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql
>
>
> +
>
> rake db:migrate
>  env RAILS_ENV=production rake db:migrate
>
>
>
> I don't understand what is wrong !
> thx,
>
>
>
>
>
>
>
>
> ___
> osmosis-dev mailing list
> osmosis-dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/osmosis-dev
>
>
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev

[osmosis-dev] error caused by Caused by: org.postgresql.util.PSQLException: ERROR: column "active" of relation "users" does not exist

2010-06-23 Thread Michael musset 
Hi,

I've a france.osm from geofabrik

i'm trying to import it into my database with osmosis with this command :

./osmosis --read-xml file="/home/mickael/planet/france.osm" --write-apidb
populateCurrentTables=yes host="localhost" database="openstreetmap"
user="openstreetmap" validateSchemaVersion=no


the osmosis come from :

wget http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-bin-latest.tgz
tar xvfz osmosis-bin-latest.tgz
cd osmosis-0.35
chmod a+x bin/osmosis
bin/osmosis


my railsport config is :
test:
  adapter: postgresql
  database: osm_test
#  username: osm_test
#  password: osm_test
  host: localhost
  encoding: utf8

production:
  adapter: postgresql
  database: osm
#  username: osm
#  password: osm
  host: localhost
  encoding: utf8

development:
  adapter: postgresql
  database: openstreetmap
#  username: openstreetmap
#  password: openstreetmap
  host: localhost
  encoding: utf8



when i've created the database i've done :

createdb -E UTF8 -O openstreetmap openstreetmap
createdb -E UTF8 -O openstreetmap osm_test
createdb -E UTF8 -O openstreetmap osm

and

psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql


+

rake db:migrate
 env RAILS_ENV=production rake db:migrate



I don't understand what is wrong !
thx,
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev