Ah, great I'll take a look this evening and see if I can add a runtime check.
On Mon, Nov 2, 2015, 5:15 PM Jochen Topf <[email protected]> wrote: > As I mentioned below: > > > > > You can test whether this bug is on your system, too: Download the > XML > > > > > for this node: http://www.openstreetmap.org/node/3382756758. Then > run > > > > > it through osmosis: > > > > > > > > > > osmosis --rx 3382756758.osm --wx out.osm > > > > > Compare the two files, you'll see the musical notation character > > > doubling > > On Mo, Nov 02, 2015 at 05:34:14 +0000, Brett Henderson wrote: > > Sorry, I'm terrible at checking this list. 6 months isn't ideal. Does > > anybody have an XML snippet that I could use for such a test? > > > > On Fri, 6 Mar 2015 at 23:50 Jochen Topf <[email protected]> wrote: > > > > > I think the bug is important and subtle enough that we should make > sure it > > > doesn't resurface again. Either by detecting the runtime or by the > check > > > you describe. At least we should put the check into a unit test, so > that > > > people who run the tests on their platform after building can be safe. > > > > > > Jochen > > > > > > On Thu, Mar 05, 2015 at 05:25:12PM +1100, Brett Henderson wrote: > > > > Date: Thu, 5 Mar 2015 17:25:12 +1100 > > > > From: Brett Henderson <[email protected]> > > > > To: Jochen Topf <[email protected]> > > > > Cc: OSM-Dev Openstreetmap <[email protected]> > > > > Subject: Re: [OSM-dev] Osmosis data corruption on Debian > Jessie/Testing > > > > > > > > I suspect that attempting to detect the underlying XML runtime would > be > > > > brittle. Another option might be to embed that bit of data in > Osmosis > > > > itself and do a self test before attempting to execute any XML tasks. > > > > > > > > I'm surprised that this is still an issue in standard Java. I tried > > > > raising tickets against Sun Java before it moved under Oracle but > never > > > got > > > > a response. I gave up, embedded Xerces in the main Osmosis > distribution, > > > > and then forgot about it. > > > > > > > > On 5 March 2015 at 10:16, Jochen Topf <[email protected]> wrote: > > > > > > > > > Hi! > > > > > > > > > > Just spent a few hours debugging this problem: The way Osmosis is > > > packaged > > > > > on Debian Jessie seems to be wrong. It doesn't use the Xerces XML > > > parser > > > > > but seems to fall back to Java default XML parser which mangles > Unicode > > > > > characters. > > > > > > > > > > This can lead to data corruption (and has for me today) when using > > > Osmosis > > > > > for planet updates etc. > > > > > > > > > > You can test whether this bug is on your system, too: Download the > XML > > > > > for this node: http://www.openstreetmap.org/node/3382756758. Then > run > > > > > it through osmosis: > > > > > > > > > > osmosis --rx 3382756758.osm --wx out.osm > > > > > > > > > > Compare the two files, you'll see the musical notation character > > > doubling > > > > > in the second case when your Osmosis is broken. The fix is simple: > Add > > > > > a line "load /usr/share/java/xercesImpl.jar" to > > > /etc/osmosis/plexus.conf. > > > > > As I understand this, it tells Java to load Xerces replacing the > > > built-in > > > > > XML parser. > > > > > > > > > > I have opened a bug with Debian. > > > > > > > > > > Arguably Osmosis should somehow detect when Xerces isn't found and > > > return > > > > > an > > > > > error instead of using a different implemenation. But I don't know > > > enough > > > > > about > > > > > Java to say whether thats possible. > > > > > > > > > > Jochen > > > > > -- > > > > > Jochen Topf [email protected] http://www.jochentopf.com/ > > > > > +49-173-7019282 > > > > > > > > > > _______________________________________________ > > > > > dev mailing list > > > > > [email protected] > > > > > https://lists.openstreetmap.org/listinfo/dev > > > > > > > > > > > -- > > > Jochen Topf [email protected] http://www.jochentopf.com/ > > > +49-173-7019282 > > > > > -- > Jochen Topf [email protected] http://www.jochentopf.com/ > +49-351-31778688 >
_______________________________________________ dev mailing list [email protected] https://lists.openstreetmap.org/listinfo/dev
