Re: [OSM-talk] Missing day replicate
On 2/11/19 12:35 PM, Jochen Topf wrote: > On Mon, Feb 11, 2019 at 12:02:23PM +0100, Rory McCann wrote: >> I've seen this error before a few times. I thought the file was bad and was >> pulling my hair out trying to find the weird bytes (which didn't exist). The >> file came from osmosis merging a few osc files together. By deleting that >> file, osmosis would make a new one and the problem went away. > > I guess its time to ditch Osmosis und switch to Osmium/PyOsmium for these > tasks... > Fully agree. This would also be good to make some progress on [1], where the lack of maintainers on osmosis side further complicates things, IMHO. [1] https://github.com/openstreetmap/operations/issues/154 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
On Mon, Feb 11, 2019 at 12:02:23PM +0100, Rory McCann wrote: > I've seen this error before a few times. I thought the file was bad and was > pulling my hair out trying to find the weird bytes (which didn't exist). The > file came from osmosis merging a few osc files together. By deleting that > file, osmosis would make a new one and the problem went away. I guess its time to ditch Osmosis und switch to Osmium/PyOsmium for these tasks... Jochen -- Jochen Topf joc...@remote.org https://www.jochentopf.com/ +49-351-31778688 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
On 09/02/2019 14:14, mmd wrote: On 2/9/19 1:58 PM, Tom Hughes wrote: On 09/02/2019 11:47, mmd wrote: For some strange reason, the gz file claims to be from "FAT filesystem", and most likely osmosis can't figure out that the file contents are in UTF-8. They all say that - presumably it's just what the Java zlib implementation puts in the header. Right, I would have guessed so, too. Once I decompress and re-compress the file on Linux, osmosis seems to be just fine with said file: I've recompressed the copy on the server now... Thanks! I'm still not clear what the best way to fix this issue would be. Probably this needs to be sorted out in osmosis somehow, though I don't know where exactly. It's quite curious that this never happened before. I've seen this error before a few times. I thought the file was bad and was pulling my hair out trying to find the weird bytes (which didn't exist). The file came from osmosis merging a few osc files together. By deleting that file, osmosis would make a new one and the problem went away. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
On 2/9/19 1:58 PM, Tom Hughes wrote: > On 09/02/2019 11:47, mmd wrote: > >> For some strange reason, the gz file claims to be from "FAT filesystem", >> and most likely osmosis can't figure out that the file contents are in >> UTF-8. > > They all say that - presumably it's just what the Java zlib > implementation puts in the header. Right, I would have guessed so, too. > >> Once I decompress and re-compress the file on Linux, osmosis seems to be >> just fine with said file: > > I've recompressed the copy on the server now... Thanks! I'm still not clear what the best way to fix this issue would be. Probably this needs to be sorted out in osmosis somehow, though I don't know where exactly. It's quite curious that this never happened before. -- ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
On 09/02/2019 11:47, mmd wrote: For some strange reason, the gz file claims to be from "FAT filesystem", and most likely osmosis can't figure out that the file contents are in UTF-8. They all say that - presumably it's just what the Java zlib implementation puts in the header. Once I decompress and re-compress the file on Linux, osmosis seems to be just fine with said file: I've recompressed the copy on the server now... Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
Am 09.02.19 um 08:17 schrieb Jochen Topf: > > this might or might not be related to the hourly change file with the > number 56107 being "strange": Osmosis can not read the file 107.osc.gz, > it reports a UTF-8 error. But if I gunzip the file, Osmosis can read the > file fine. So this points to an error in Osmosis' handling gzip'ed > files. Unfortunately Osmosis is no longer maintained. > For some strange reason, the gz file claims to be from "FAT filesystem", and most likely osmosis can't figure out that the file contents are in UTF-8. Once I decompress and re-compress the file on Linux, osmosis seems to be just fine with said file: $ wget https://planet.openstreetmap.org/replication/hour/000/056/107.osc.gz $ file 107.osc.gz 107.osc.gz: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT) $ ./osmosis --read-xml-change file=107.osc.gz --write-xml-change file=107.osc.new ... Caused by: org.xml.sax.SAXParseException; lineNumber: 243521; columnNumber: 1; Invalid byte 2 of 4-byte UTF-8 sequence. ... $ gunzip 107.osc.gz $ gzip 107.osc $ file 107.osc.gz 107.osc.gz: gzip compressed data, was "107.osc", last modified: Wed Feb 6 02:02:08 2019, from Unix $ ./osmosis --read-xml-change file=107.osc.gz --write-xml-change file=107.osc.new -> works fine -- ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
Hi! this might or might not be related to the hourly change file with the number 56107 being "strange": Osmosis can not read the file 107.osc.gz, it reports a UTF-8 error. But if I gunzip the file, Osmosis can read the file fine. So this points to an error in Osmosis' handling gzip'ed files. Unfortunately Osmosis is no longer maintained. Generally errors like this should probably be reported on https://trac.openstreetmap.org/ or the dev mailing list or #osm-dev IRC channel would be a good place. Jochen On Sat, Feb 09, 2019 at 11:49:45AM +1100, Andrew Davidson wrote: > Date: Sat, 9 Feb 2019 11:49:45 +1100 > From: Andrew Davidson > To: talk@openstreetmap.org > Subject: Re: [OSM-talk] Missing day replicate > > Hasn't worked for three days now: > > https://munin.openstreetmap.org/problems.html#critical > > also the Planet dump is also not working. I'm not sure how we're supposed to > report these, there used to be a status page on the wikki but that appears > to no longer be the appropriate place. > > On 8/2/19 8:22 am, Andre Hinrichs wrote: > > Hi list, > > > > I am missing the day replicate for today (2019-02-07)... > > > > Hour and minute replicate seems to work ok. > > > > Please check the process... > > > > > > Greetings > > > > Andre > > > > > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > > https://munin.openstreetmap.org/problems.html#critical > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk -- Jochen Topf joc...@remote.org https://www.jochentopf.com/ +49-351-31778688 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Missing day replicate
Hasn't worked for three days now: https://munin.openstreetmap.org/problems.html#critical also the Planet dump is also not working. I'm not sure how we're supposed to report these, there used to be a status page on the wikki but that appears to no longer be the appropriate place. On 8/2/19 8:22 am, Andre Hinrichs wrote: Hi list, I am missing the day replicate for today (2019-02-07)... Hour and minute replicate seems to work ok. Please check the process... Greetings Andre ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk https://munin.openstreetmap.org/problems.html#critical ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
