Re: [Python-ideas] Importing public symbols and simultainiously privatizing them, is too noisy

[Abe Dillon]

>
> I avoid  __all__ like the plague. Too easy for it to get out of sync with
> the API when i forget to add a new symbol.


Your API should be one of the most stable parts of your code, no?

On Fri, Mar 18, 2016 at 4:29 PM, Chris Barker  wrote:

> On Wed, Mar 16, 2016 at 6:52 PM, Rick Johnson <
> rantingrickjohn...@gmail.com> wrote:
>
>> > Besides, why is "import x as _x" so special to require special syntax?
>>
>
> It's not :-) I know I do, for instance,
>
> from matplotlib  import pylot as plt
>
> But have NEVER done the leading underscore thing...
>
>
>>  from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam,
>> eggs as _eggs
>>
>
> if you are mirroring an entire namespace, or a god fraction of one then
> use a module name!
>
> import module as _mod
>
> then use _mod.Foo, etc.
>
> Now, that may seem like a contrived example, but i've
>> witnessed much longer "run-on import lines" than that.
>>
>
> I have too, but I think it's  bad style -- if you are importing a LOT of
> names from one module, just import the darn module -- giving it a shorter
> name if you like. This has become a really standard practice, like:
>
> import numpy as np
>
> for instance.
>
> The intended purpose is to: "automate the privatization of
>> public symbols during the import process".
>>
>
> I'm really confused about the use case for "privatization of public
> symbols" at all, but again, if you need a lot of them, use the module name
> to prefix them. Heck give it a one character name, and then it's hardly
> more typing than the underscore...
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] VT100 style escape codes in Windows

[Random832]

On Wed, Dec 28, 2016, at 18:33, Joseph Hackman wrote:
> The quick answer is that the MSDN doc indicates support from windows 2000
> onward, with no notes for partial compatability:
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms686033(v=vs.85).aspx

That's the function itself (and 2000 is just as far back as the website
goes, it's actually existed, with the other modes, since NT 3.1 and
Windows 95.

The separate code sample page mentions that they are new features since
Windows 10 Anniversary Edition.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] VT100 style escape codes in Windows

[Joseph Hackman]

Welp! You're definitely correct. Ah well.

On 28 December 2016 at 18:33, Joseph Hackman 
wrote:

> The quick answer is that the MSDN doc indicates support from windows 2000
> onward, with no notes for partial compatability:
> https://msdn.microsoft.com/en-us/library/windows/desktop/
> ms686033(v=vs.85).aspx
>
> I'll build a Windows 7 VM to test.
>
> I believe Python 3.6 is only supported on Vista+ and 3.7 would be Windows
> 7+ only?
>
> On 28 December 2016 at 18:06, Paul Moore  wrote:
>
>> Would this only apply to recent versions of Windows? (IIRC, the VT100
>> support is Win10 only). If so, I'd be concerned about scripts that
>> worked on *some* Windows versions but not others. And in particular,
>> about scripts written on Unix using raw VT codes rather than using a
>> portable solution like colorama.
>>
>> At the point where we can comfortably assume the majority of users are
>> using a version of Windows that supports VT codes, I'd be OK with it
>> being the default, but until then I'd prefer it were an opt-in option.
>> Paul
>>
>> On 28 December 2016 at 23:00, Joseph Hackman 
>> wrote:
>> > Hey All!
>> >
>> > I propose that Windows CPython flip the bit for VT100 support (colors
>> and
>> > whatnot) for the stdout/stderr streams at startup time.
>> >
>> > I believe this behavior is worthwhile because ANSI escape codes are
>> standard
>> > across most of Python's install base, and the alternative for Windows
>> (using
>> > ctypes/win32 to alter the colors) is non-intuitive and well beyond the
>> scope
>> > of most users.
>> >
>> > Under Linux/Mac, the terminal always supports what it can, and it's up
>> to
>> > the application to verify escape codes are supported. Under Windows,
>> > applications (Python) must specifically request that escape codes be
>> > enabled. The flag lasts for the duration of the application, and must be
>> > flipped on every launch. It seems many of the built-in windows commands
>> now
>> > operate in this mode.
>> >
>> > This change would not impede tools that use the win32 APIs for the
>> console
>> > (such as colorama), and is supported in windows 2000 and up.
>> >
>> > The only good alternatives I can see is adding colorized/special output
>> as a
>> > proper python feature that actually checks using the terminal
>> information in
>> > *nix and win32.
>> >
>> > For more info, please see the issue: http://bugs.python.org/issue29059
>> >
>> > Cheers,
>> > Joseph
>> >
>> >
>> >
>> > ___
>> > Python-ideas mailing list
>> > Python-ideas@python.org
>> > https://mail.python.org/mailman/listinfo/python-ideas
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] VT100 style escape codes in Windows

[Joseph Hackman]

The quick answer is that the MSDN doc indicates support from windows 2000
onward, with no notes for partial compatability:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms686033(v=vs.85).aspx

I'll build a Windows 7 VM to test.

I believe Python 3.6 is only supported on Vista+ and 3.7 would be Windows
7+ only?

On 28 December 2016 at 18:06, Paul Moore  wrote:

> Would this only apply to recent versions of Windows? (IIRC, the VT100
> support is Win10 only). If so, I'd be concerned about scripts that
> worked on *some* Windows versions but not others. And in particular,
> about scripts written on Unix using raw VT codes rather than using a
> portable solution like colorama.
>
> At the point where we can comfortably assume the majority of users are
> using a version of Windows that supports VT codes, I'd be OK with it
> being the default, but until then I'd prefer it were an opt-in option.
> Paul
>
> On 28 December 2016 at 23:00, Joseph Hackman 
> wrote:
> > Hey All!
> >
> > I propose that Windows CPython flip the bit for VT100 support (colors and
> > whatnot) for the stdout/stderr streams at startup time.
> >
> > I believe this behavior is worthwhile because ANSI escape codes are
> standard
> > across most of Python's install base, and the alternative for Windows
> (using
> > ctypes/win32 to alter the colors) is non-intuitive and well beyond the
> scope
> > of most users.
> >
> > Under Linux/Mac, the terminal always supports what it can, and it's up to
> > the application to verify escape codes are supported. Under Windows,
> > applications (Python) must specifically request that escape codes be
> > enabled. The flag lasts for the duration of the application, and must be
> > flipped on every launch. It seems many of the built-in windows commands
> now
> > operate in this mode.
> >
> > This change would not impede tools that use the win32 APIs for the
> console
> > (such as colorama), and is supported in windows 2000 and up.
> >
> > The only good alternatives I can see is adding colorized/special output
> as a
> > proper python feature that actually checks using the terminal
> information in
> > *nix and win32.
> >
> > For more info, please see the issue: http://bugs.python.org/issue29059
> >
> > Cheers,
> > Joseph
> >
> >
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] VT100 style escape codes in Windows

[Paul Moore]

Would this only apply to recent versions of Windows? (IIRC, the VT100
support is Win10 only). If so, I'd be concerned about scripts that
worked on *some* Windows versions but not others. And in particular,
about scripts written on Unix using raw VT codes rather than using a
portable solution like colorama.

At the point where we can comfortably assume the majority of users are
using a version of Windows that supports VT codes, I'd be OK with it
being the default, but until then I'd prefer it were an opt-in option.
Paul

On 28 December 2016 at 23:00, Joseph Hackman  wrote:
> Hey All!
>
> I propose that Windows CPython flip the bit for VT100 support (colors and
> whatnot) for the stdout/stderr streams at startup time.
>
> I believe this behavior is worthwhile because ANSI escape codes are standard
> across most of Python's install base, and the alternative for Windows (using
> ctypes/win32 to alter the colors) is non-intuitive and well beyond the scope
> of most users.
>
> Under Linux/Mac, the terminal always supports what it can, and it's up to
> the application to verify escape codes are supported. Under Windows,
> applications (Python) must specifically request that escape codes be
> enabled. The flag lasts for the duration of the application, and must be
> flipped on every launch. It seems many of the built-in windows commands now
> operate in this mode.
>
> This change would not impede tools that use the win32 APIs for the console
> (such as colorama), and is supported in windows 2000 and up.
>
> The only good alternatives I can see is adding colorized/special output as a
> proper python feature that actually checks using the terminal information in
> *nix and win32.
>
> For more info, please see the issue: http://bugs.python.org/issue29059
>
> Cheers,
> Joseph
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] VT100 style escape codes in Windows

[Joseph Hackman]

Hey All!

I propose that Windows CPython flip the bit for VT100 support (colors and
whatnot) for the stdout/stderr streams at startup time.

I believe this behavior is worthwhile because ANSI escape codes are
standard across most of Python's install base, and the alternative for
Windows (using ctypes/win32 to alter the colors) is non-intuitive and well
beyond the scope of most users.

Under Linux/Mac, the terminal always supports what it can, and it's up to
the application to verify escape codes are supported. Under Windows,
applications (Python) must specifically request that escape codes be
enabled. The flag lasts for the duration of the application, and must be
flipped on every launch. It seems many of the built-in windows commands now
operate in this mode.

This change would not impede tools that use the win32 APIs for the console
(such as colorama), and is supported in windows 2000 and up.

The only good alternatives I can see is adding colorized/special output as
a proper python feature that actually checks using the terminal information
in *nix and win32.

For more info, please see the issue: http://bugs.python.org/issue29059

Cheers,
Joseph
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Function arguments in tracebacks

[Mahmoud Hashemi]

On Wed, Dec 28, 2016 at 2:13 PM, Nathaniel Smith  wrote:

> On Dec 28, 2016 12:44, "Brett Cannon"  wrote:
>
> My quick on-vacation response is that attaching more objects to exceptions
> is typically viewed as dangerous as it can lead to those objects being kept
> alive longer than expected (see the discussions about richer error messages
> to see that worry come out for something as simple as attaching the type to
> a TypeError).
>
>
> This isn't an issue for printing arguments or other locals in tracebacks,
> though. The traceback printing code can access anything in the frame stack.
>
> -n
>
>
Right. I'd actually be more worried about security leaks than memory leaks.
Imagine you're calling a password checking function that got bytes instead
of text, what amounts to a type check could leak the plaintext password.
One rarely sees a C traceback, let alone a textual one, except during
development, whereas Python tracebacks are seen during development and
after deployment.

Mahmoud
https://github.com/mahmoud
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Function arguments in tracebacks

[Nathaniel Smith]

On Dec 28, 2016 12:44, "Brett Cannon"  wrote:

My quick on-vacation response is that attaching more objects to exceptions
is typically viewed as dangerous as it can lead to those objects being kept
alive longer than expected (see the discussions about richer error messages
to see that worry come out for something as simple as attaching the type to
a TypeError).


This isn't an issue for printing arguments or other locals in tracebacks,
though. The traceback printing code can access anything in the frame stack.

-n
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[Ned Batchelder]

On 12/28/16 12:27 PM, j...@math.brown.edu wrote:
> On Wed, Dec 28, 2016 at 11:48 AM, Ned Batchelder
> mailto:n...@nedbatchelder.com>> wrote:
>
> You can write a simple function to use hash iteratively to hash
> the entire stream in constant space and linear time:
>
> def hash_stream(them):
> val = 0
> for it in them:
> val = hash((val, it))
> return val
>
> Although this creates N 2-tuples, they come and go, so the memory
> use won't grow.  Adjust the code as needed to achieve
> canonicalization before iterating.
>
> Or maybe I am misunderstanding the requirements?
>
>
> This is better than solutions like
> http://stackoverflow.com/a/27952689/161642
>  in the sense that it's a
> little higher level (no bit shifting or magic numbers).
>
> But it's not clear that it's any better in the sense that you're still
> rolling your own incremental hash algorithm out of a lower-level
> primitive that doesn't document support for this, and therefore taking
> responsibility yourself for how well it distributes values into buckets.
>
> Are you confident this results in good hash performance? Is this
> better than a solution built on top of a hash function with an
> explicit API for calculating a hash incrementally, such as the crc32
> example I included? (And again, this would ideally be
> a sys.hash_info.hash_bits -bit algorithm.)

I don't have the theoretical background to defend this function.  But it
seems to me that if we believe that hash((int, thing)) distributes well,
then how could this function not distribute well?

--Ned.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Function arguments in tracebacks

[MRAB]

On 2016-12-28 21:01, Emanuel Landeholm wrote:

I think an argument could be made for including the str() of parameters
of primitive types and with small values (for some value of "primitive"
and "small", can of worms here...). I'm thinking numbers and short
strings. Maybe a flag to control this behaviour? My gut feeling is that
this would be a hack with lots of corner cases and surprises so it would
probably not be very helpful in the general case.


Don't you mean the repr or ascii because you'll want 'foo' to print as:

'foo'

and not as:

foo

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Function arguments in tracebacks

[Emanuel Landeholm]

I think an argument could be made for including the str() of parameters of
primitive types and with small values (for some value of "primitive" and
"small", can of worms here...). I'm thinking numbers and short strings.
Maybe a flag to control this behaviour? My gut feeling is that this would
be a hack with lots of corner cases and surprises so it would probably not
be very helpful in the general case.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Function arguments in tracebacks

[Brett Cannon]

My quick on-vacation response is that attaching more objects to exceptions
is typically viewed as dangerous as it can lead to those objects being kept
alive longer than expected (see the discussions about richer error messages
to see that worry come out for something as simple as attaching the type to
a TypeError).

On Tue, 27 Dec 2016 at 09:26 Ammar Askar  wrote:

> Consider the following similar C and Python code and their
> tracebacks:
>
> C
> ---
>   int divide(int x, int y, char* some_string) {
>   return x / y;
>   }
>   int main(...) {
>   divide(2, 0, "Hello World");
>   }
> ---
>   Program received signal SIGFPE, Arithmetic exception.
>   (gdb) bt
>   #0  0x004004c4 in divide (x=2, y=0, some_string=0x4005a8
> "Hello World") at test.c:2
>   #1  0x004004e7 in main (argc=1, argv=0x7fffe328) at test.c:6
>
> Python
> ---
>   def divide(x, y, some_string):
> return x / y
>
>   divide(2, 0, "Hello World")
> ---
>   Traceback (most recent call last):
> File "test.py", line 4, in 
> File "test.py", line 2, in divide
>   ZeroDivisionError: division by zero
>
>
> By including the function arguments within the traceback, we
> can get more information at a glance than we could with just
> the names of methods.
>
> This would be pretty cool and stop the occasional "printf"
> debugging without cluttering up the traceback too much.
>
> There will definitely need to be some reasonable line length
> limit  because the repr() of parameters could be really long.
> In similar situations gdb replaces the value in the traceback
> with elipsis, and I believe that's a good solution for python
> as well.
>
> Obviously this isn't a great example since the error is immediately
> obvious but I think this could be potentially useful in a bunch
> of situations.
>
> I've made a a quick toy implementation in traceback.c, this is what
> it looks like for the script above.
>
>   Traceback (most recent call last):
> File "test.py", line 4, in 
>   divide(2, 0, "Hello World")
> File "test.py", line 2, in divide (x=2, y=0, some_string='Hello World')
>   return x / y
>   ZeroDivisionError: division by zero
>
>
> == Potential Downsides ==
>
> There's probably a lot more than these, but I could only think of
> these so far.
>
> * Private data might be leaked, imagine a
>
> def login(username, password):
> ...
>
>   method. While function names/source files/source code are also
>   private, variables can potentially contain all kinds of sensitive data.
>
> * A variable that takes a long time to return a string representation may
>   significantly slow down the time it takes to generate a traceback.
>
> * We can really only return the state of the variables when the
>   traceback is printed, this might result in some slightly un-intuitive
>   behavior. (Easier to explain with an example)
>
> def f(x):
>x = 2
>raise Exception()
>
> f(1)
>
>   Traceback (most recent call last):
> File "", line 1, in 
> File "", line 3, in f(x=2)
>
>   The fact that x is mutated within the function body means that the
> value printed in
>   the traceback is the changed value which might be slightly misleading.
>
>
> I'd love to hear your guy's thoughts on the idea.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[jab]

On Wed, Dec 28, 2016 at 12:10 PM, Ethan Furman  wrote:

> In other words, objects that do not compare equal can also have the same
> hash value (although too much of that will reduce the efficiency of
> Python's containers).
>

Yes, I realize that unequal objects can return the same hash value with
only performance, and not correctness, suffering. It's the performance I'm
concerned about. That's what I meant by "...to keep from unnecessarily
causing hash collisions..." in my original message, but sorry this wasn't
clearer. We should be able to do this in a way that doesn't increase hash
collisions unnecessarily.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[jab]

On Wed, Dec 28, 2016 at 11:48 AM, Ned Batchelder 
wrote:

> You can write a simple function to use hash iteratively to hash the entire
> stream in constant space and linear time:
>
> def hash_stream(them):
> val = 0
> for it in them:
> val = hash((val, it))
> return val
>
> Although this creates N 2-tuples, they come and go, so the memory use
> won't grow.  Adjust the code as needed to achieve canonicalization before
> iterating.
>
> Or maybe I am misunderstanding the requirements?
>

This is better than solutions like http://stackoverflow.com/a/
27952689/161642 in the sense that it's a little higher level (no bit
shifting or magic numbers).

But it's not clear that it's any better in the sense that you're still
rolling your own incremental hash algorithm out of a lower-level primitive
that doesn't document support for this, and therefore taking responsibility
yourself for how well it distributes values into buckets.

Are you confident this results in good hash performance? Is this better
than a solution built on top of a hash function with an explicit API for
calculating a hash incrementally, such as the crc32 example I included?
(And again, this would ideally be a sys.hash_info.hash_bits -bit algorithm.)

Don't we still probably want either:

1) Python to provide some such hash_stream() function as a built-in,

or failing that,

2) the https://docs.python.org/3/reference/datamodel.html#object.__hash__
documentation to bless this as the recommended solution to this problem,
thereby providing assurance of its performance?

If that makes sense, I'd be happy to file an issue, and include the start
of a patch providing either 1 or 2.

Thanks very much for the helpful response.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[Ethan Furman]

On 12/27/2016 07:13 PM, j...@math.brown.edu wrote:


According to the docs [6]:

"""
it is advised to mix together the hash values of the components of the
object that also play a part in comparison of objects by packing them
into a tuple and hashing the tuple. Example:

def __hash__(self):
 return hash((self.name, self.nick, self.color))

"""


Applying this advice to the use cases above would require creating an
arbitrarily large tuple in memory before passing it to hash(), which
is then just thrown away. It would be preferable if there were a way
to pass multiple values to hash() in a streaming fashion, such that
the overall hash were computed incrementally, without building up a
large object in memory first.


Part of the reason for creating __hash__ like above is that:

- it's simple
- it's reliable

However, it's not the only way to have a hash algorithm that works; in fact, 
the beginning of the sentence you quoted says:


The only required property is that objects which compare equal have
 the same hash value;


In other words, objects that do not compare equal can also have the same hash 
value (although too much of that will reduce the efficiency of Python's 
containers).


(ii) priming the overall hash value with some class-specific initial
value, so that if an instance of a different type of collection, which
comprised the same items but which compared unequal, were to compute
its hash value out of the same constituent items, we make sure our
hash value differs.


This is unnecessary:  hashes are compared first as a way to weed out impossible 
matches, but when the hashes are the same an actual __eq__ test is still done 
[7].

--
~Ethan~


[6] https://docs.python.org/3/reference/datamodel.html#object.__hash__
[7] some test code to prove above points:

--- 8< 
from unittest import main, TestCase

class Eggs(object):
def __init__(self, value):
self.value = value
def __hash__(self):
return hash(self.value)
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.value == other.value
return NotImplemented

class Spam(object):
def __init__(self, value):
self.value = value
def __hash__(self):
return hash(self.value)
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.value == other.value
return NotImplemented

e10 = Eggs(1)
e20 = Eggs(2)
e11 = Eggs(1)
e21 = Eggs(2)

s10 = Spam(1)
s20 = Spam(2)
s11 = Spam(1)
s21 = Spam(2)

bag = {}
bag[e10] = 1
bag[s10] = 2
bag[e20] = 3
bag[s20] = 4

class TestEqualityAndHashing(TestCase):

def test_equal(self):
# same class, same value --> equal
self.assertEqual(e10, e11)
self.assertEqual(e20, e21)
self.assertEqual(s10, s11)
self.assertEqual(s20, s21)

def test_not_equal(self):
# different class, same value --> not equal
self.assertEqual(e10.value, s10.value)
self.assertNotEqual(e10, s10)
self.assertEqual(e20.value, s20.value)
self.assertNotEqual(e20, s20)

def test_same_hash(self):
# same class, same value, same hash
self.assertEqual(hash(e10), hash(e11))
self.assertEqual(hash(e20), hash(e21))
self.assertEqual(hash(s10), hash(s11))
self.assertEqual(hash(s20), hash(s21))

# different class, same value, same hash
self.assertEqual(hash(e10), hash(s10))
self.assertEqual(hash(e11), hash(s11))
self.assertEqual(hash(e20), hash(s20))
self.assertEqual(hash(e21), hash(s21))

def test_as_key(self):
# different objects from different classes with same hash should still 
be distinct
self.assertEqual(len(bag), 4)
self.assertEqual(bag[e10], 1)
self.assertEqual(bag[s10], 2)
self.assertEqual(bag[e20], 3)
self.assertEqual(bag[s20], 4)

# different objects from same classes with same hash should not be 
distinct
self.assertEqual(bag[e11], 1)
self.assertEqual(bag[s11], 2)
self.assertEqual(bag[e21], 3)
self.assertEqual(bag[s21], 4)

main()
--- 8< 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[Ned Batchelder]

On 12/27/16 10:13 PM, j...@math.brown.edu wrote:
> Applying this advice to the use cases above would require creating an
> arbitrarily large tuple in memory before passing it to hash(), which
> is then just thrown away. It would be preferable if there were a way
> to pass multiple values to hash() in a streaming fashion, such that
> the overall hash were computed incrementally, without building up a
> large object in memory first.
>
> Should there be better support for this use case? Perhaps hash() could
> support an alternative signature, allowing it to accept a stream of
> values whose combined hash would be computed incrementally in
> *constant* space and linear time, e.g. "hash(items=iter(self))".
You can write a simple function to use hash iteratively to hash the
entire stream in constant space and linear time:

def hash_stream(them):
val = 0
for it in them:
val = hash((val, it))
return val

Although this creates N 2-tuples, they come and go, so the memory use
won't grow.  Adjust the code as needed to achieve canonicalization
before iterating.

Or maybe I am misunderstanding the requirements?

--Ned.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] incremental hashing in __hash__

[jab]

I actually have been poking around that code already. I also found
https://github.com/vperron/python-superfasthash/blob/master/superfasthash.py
in case of interest.

But it still seems like library authors with this use case should keep
their library code free of implementation details like this, and instead
use a higher-level API provided by Python.

Thanks,
Josh

On Tue, Dec 27, 2016 at 10:28 PM, Ryan Gonzalez  wrote:

> You could always try to make a Python version of the C tuple hashing
> function[1] (requires the total # of elements) or PyPy's[2] (seems like it
> would allow true incremental hashing). API idea:
>
>
> hasher = IncrementalHasher()
> hasher.add(one_item_to_hash)  # updates hasher.hash property with result
> # repeat
> return hasher.hash
>
>
> [1]: https://hg.python.org/cpython/file/dcced3bd22fe/
> Objects/tupleobject.c#l331
> [2]: https://bitbucket.org/pypy/pypy/src/d8febc18447e1f785a384d52413a34
> 5d7b3db423/rpython/rlib/objectmodel.py#objectmodel.py-562
>
> --
> Ryan (ライアン)
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
> http://kirbyfan64.github.io/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/