New submission from josch:
I recently realized that the output of the following is different between 32
bit and 64 bit architectures:
PYTHONHASHSEED=0 python3 -c 'print(hash("a"))'
In my case, I'm running some test cases which involve calling a Python module
which creates several hundred megabyte big graphs and other things. The fastest
way to make sure that the output I get is the same that I expect is to just
call the md5sum or sha256sum shell tools on the output and compare them with
the expected values. Unfortunately, some libraries I use rely on the order of
items in Python dictionaries for their output. Yes, they should not do that but
they also don't care and thus don't fix the problem.
My initial solution to this was to use PYTHONHASHSEED=0 which helped but I now
found out that this is limited to producing the same hash within the set of 32
bit and 64 bit architectures, respectively. See above line which behaves
different depending on the integer size of architectures.
So what I'd like CPython to have is yet another workaround like PYTHONHASHSEED
which allows me to temporarily influence the inner workings of the hash()
function such that it behaves the same on 32 bit and 64 bit architectures.
Maybe something like PYTHONHASH32BIT or similar?
If I understand the CPython hash function correctly, then this environment
variable would just bitmask the result of the function with 0xFFFFFFFF or cast
it to int32_t to achieve the same output across architectures.
Would this be possible?
My only alternative seems to be to either maintain patched versions of all
modules I use which wrongly rely on dictionary ordering or to go to great
lengths of parsing the (more or less) random output they produce into a sorted
intermediary format - which seems like a bad idea because the files are several
hundred megabytes big and this would just take very long and require additional
complexity in handling them compared to being able to just md5sum or sha256sum
them for the sake of checking whether my test cases succeed or not.
----------
messages: 229219
nosy: josch
priority: normal
severity: normal
status: open
title: Please make it possible to make the output of hash() equal between 32
and 64 bit architectures
versions: Python 3.5
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue22621>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
