Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Terry Reedy
On 3/21/2019 1:23 PM, Paul Moore wrote: On Thu, 21 Mar 2019 at 17:05, Steve Holden wrote: Especially as the standards specifically say that ordering has no semantic impact. Byte-by-byte comparison of XML is almost always inappropriate. Conversely, if ordering has no semantic impact,

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Stefan Behnel
Victor Stinner schrieb am 21.03.19 um 01:22: > Alternatives have been proposed like a recipe to sort node attributes > before serialization, but honestly, it's way too complex. Hm, really? Five lines of simple and obvious Python code, that provide a fast and completely Python-version agnostic

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Jonathan Goble
On Thu, Mar 21, 2019, 1:05 PM Steve Holden wrote: > On Thu, Mar 21, 2019 at 11:33 AM Antoine Pitrou > wrote: > >> [...] >> >> Most users and applications should /never/ care about the order of XML >> attributes. >> >> Regards >> >> Antoine >> > > Especially as the standards specifically say

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Paul Moore
On Thu, 21 Mar 2019 at 17:05, Steve Holden wrote: > > On Thu, Mar 21, 2019 at 11:33 AM Antoine Pitrou wrote: >> >> [...] >> >> Most users and applications should /never/ care about the order of XML >> attributes. >> >> Regards >> >> Antoine > > > Especially as the standards specifically say that

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Steve Holden
On Thu, Mar 21, 2019 at 11:33 AM Antoine Pitrou wrote: > [...] > > Most users and applications should /never/ care about the order of XML > attributes. > > Regards > > Antoine > Especially as the standards specifically say that ordering has no semantic impact. Byte-by-byte comparison of XML is

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Antoine Pitrou
On Thu, 21 Mar 2019 02:07:01 +0100 Victor Stinner wrote: > Le lun. 18 mars 2019 à 23:41, Raymond Hettinger > a écrit : > > The code in the current 3.8 alpha differs from 3.7 in that it removes > > attribute sorting and instead preserves the order the user specified when > > creating an

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Raymond Hettinger
> On Mar 20, 2019, at 6:07 PM, Victor Stinner wrote: > > what's the rationale of this backward incompatible change? Please refrain from abusive mischaracterizations. It is only backwards incompatible if there was a guaranteed behavior. Whether there was or not is what this thread is

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Victor Stinner
Le lun. 18 mars 2019 à 23:41, Raymond Hettinger a écrit : > The code in the current 3.8 alpha differs from 3.7 in that it removes > attribute sorting and instead preserves the order the user specified when > creating an element. As far as I can tell, there is no objection to this as > a

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Raymond Hettinger
> On Mar 20, 2019, at 5:22 PM, Victor Stinner wrote: > > I don't understand why such simple solution has been rejected. It hasn't been rejected. That is above my pay grade. Stefan and I recommended against going down this path. However, since you're in disagreement and have marked this as

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Victor Stinner
Le jeu. 21 mars 2019 à 01:30, Raymond Hettinger a écrit : > There's no preaching and no judgment. We can't have a conversation though if > we can't state the crux of the problem: some existing tests in third-party > modules depend on the XML serialization being byte-for-byte identical >

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Raymond Hettinger
> On Mar 19, 2019, at 4:53 AM, Ned Batchelder wrote: > > None of this is impossible, but please try not to preach to us maintainers > that we are doing it wrong, that it will be easy to fix, etc There's no preaching and no judgment. We can't have a conversation though if we can't state the

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-20 Thread Victor Stinner
Hi, Le lun. 18 mars 2019 à 23:41, Raymond Hettinger a écrit : > We're having a super interesting discussion on > https://bugs.python.org/issue34160 . It is now marked as a release blocker > and warrants a broader discussion. Thanks for starting a thread on python-dev. I'm the one who raised

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Gregory P. Smith
On Tue, Mar 19, 2019, 4:53 AM Ned Batchelder wrote: > On 3/19/19 4:13 AM, Serhiy Storchaka wrote: > > 19.03.19 00:41, Raymond Hettinger пише: > >> 4) Fix the tests in the third-party modules to be more focused on > >> their actual test objectives, the semantics of the generated XML > >> rather

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Tim Delaney
On Wed, 20 Mar 2019 at 00:29, Serhiy Storchaka wrote: > 19.03.19 15:10, Tim Delaney пише: > > Now Calibre is definitely in the wrong here - it should be able to > > import regardless of the order of attributes. But the fact is that there > > are a lot of tools out there that are semi-broken in a

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Stefan Behnel
Ned Batchelder schrieb am 19.03.19 um 12:53: > I need to re-engineer my tests. … or sort the attributes before serialisation, or use C14N always, or change your code to create the attributes in sorted-by-name order. The new behaviour allows for a couple of ways to deal with the issue of backwards

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Stefan Behnel
Nathaniel Smith schrieb am 19.03.19 um 00:15: > That seems potentially simpler to implement than canonical XML > serialization C14N is already implemented for ElementTree, just needs to be ported to Py3.8 and merged. https://bugs.python.org/issue13611 Stefan

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Brett Cannon
On Tue, Mar 19, 2019 at 6:15 AM Serhiy Storchaka wrote: > 19.03.19 13:53, Ned Batchelder пише: > > Option 4 is misleading. Is anyone here really offering to "fix the > > tests in third-party modules"? Option 4 is actually, "do nothing, and > > let a multitude of projects figure out how to fix

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Stéphane Wirtel
and why not with a very long PendingDeprecationWarning? this warning could be used in this case. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Serhiy Storchaka
19.03.19 15:10, Tim Delaney пише: Now Calibre is definitely in the wrong here - it should be able to import regardless of the order of attributes. But the fact is that there are a lot of tools out there that are semi-broken in a similar manner. Is not Calibre going to seat on Python 2

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Serhiy Storchaka
19.03.19 14:50, Antoine Pitrou пише: 2). Go into every XML module and add attribute sorting options to each function that generate xml. What do you mean with "every XML module"? Are there many of them? ElementTree and minidom. Maybe xmlrpc. And perhaps we need to add arguments in calls at

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Serhiy Storchaka
19.03.19 13:53, Ned Batchelder пише: Option 4 is misleading.  Is anyone here really offering to "fix the tests in third-party modules"?  Option 4 is actually, "do nothing, and let a multitude of projects figure out how to fix their tests, slowing progress in those projects as they try to

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Tim Delaney
On Tue, 19 Mar 2019 at 23:13, David Mertz wrote: > In a way, this case makes bugs worse because they are not only a Python > internal matter. XML is used to communicate among many tools and > programming languages, and relying on assumptions those other tools will > not follow us a bad habit. >

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Antoine Pitrou
Hi Raymond, As long as the new serialization order is deterministic (i.e. it's the same every run and doesn't depend on e.g. hash randomization), then I think it's fine to change it. Some more comments / questions: > 2). Go into every XML module and add attribute sorting options to each >

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread David Mertz
In my opinion, any test that relied on a non-promised accident of serialization is broken today. Very often, such bad tests mask bad production code that makes the same unsafe assumptions. This is similar to tests that assumed a certain dictionary order, before we got guaranteed insertion order.

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Ned Batchelder
On 3/19/19 4:13 AM, Serhiy Storchaka wrote: 19.03.19 00:41, Raymond Hettinger пише: 3) Add a standards compliant canonicalization tool (see https://en.wikipedia.org/wiki/Canonical_XML ).  This is likely to be the right-way-to-do-it but takes time and energy. 4) Fix the tests in the

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Serhiy Storchaka
19.03.19 00:41, Raymond Hettinger пише: 3) Add a standards compliant canonicalization tool (see https://en.wikipedia.org/wiki/Canonical_XML ). This is likely to be the right-way-to-do-it but takes time and energy. 4) Fix the tests in the third-party modules to be more focused on their actual

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Gregory P. Smith
On Mon, Mar 18, 2019 at 9:44 PM Terry Reedy wrote: > On 3/18/2019 6:41 PM, Raymond Hettinger wrote: > > We're having a super interesting discussion on > https://bugs.python.org/issue34160 . It is now marked as a release > blocker and warrants a broader discussion. > > > > Our problem is that at

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Terry Reedy
On 3/18/2019 6:41 PM, Raymond Hettinger wrote: We're having a super interesting discussion on https://bugs.python.org/issue34160 . It is now marked as a release blocker and warrants a broader discussion. Our problem is that at least two distinct and important users have written tests that

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Raymond Hettinger
> On Mar 18, 2019, at 4:15 PM, Nathaniel Smith wrote: > > I noticed that your list doesn't include "add a DOM equality operator". That > seems potentially simpler to implement than canonical XML serialization, and > like a useful thing to have in any case. Would it make sense as an option?

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Nathaniel Smith
I noticed that your list doesn't include "add a DOM equality operator". That seems potentially simpler to implement than canonical XML serialization, and like a useful thing to have in any case. Would it make sense as an option? On Mon, Mar 18, 2019, 15:46 Raymond Hettinger wrote: > We're

[Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Raymond Hettinger
We're having a super interesting discussion on https://bugs.python.org/issue34160 . It is now marked as a release blocker and warrants a broader discussion. Our problem is that at least two distinct and important users have written tests that depend on exact byte-by-byte comparisons of the