[issue11771] hashlib object cannot be pickled

[Andrey Kislyuk]

Andrey Kislyuk added the comment:

For anyone else looking for a solution to this, I wrote a library: 
https://github.com/kislyuk/rehash

--
nosy: +Andrey.Kislyuk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Klaus Wolf]

Klaus Wolf added the comment:

Please reopen this bug. To answer the question: Why on Earth would you want to 
serialize a hashlib object? : multiprocessing.connection.ForkingPickler wants. 
I.e. if you want to parallelize your hash calculations, this will obstruct your 
efforts.

--
nosy: +approximately

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Gregory P. Smith]

Gregory P. Smith added the comment:

Do you honestly have a situation where you need to share a computationally
significant amount of hashing state only to want to finish the computation
N different times with alternate computationally significant ending data
that multiprocessing would actually help with where you cannot use
threads?  Hashlib releases the GIL during nontrivial hash computations.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Klaus Wolf]

Klaus Wolf added the comment:

You want to say: It doesn't work, but it is somehow intentional because you 
never used id, correct?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Gregory P. Smith]

Gregory P. Smith added the comment:

Please be constructive.

There is no way to implement generic pickling for hash objects that would work 
across all implementations.  

The underlying code implementing each function is free to store its internal 
state however it wants and does not provide an API to get at it or any standard 
representation of it.

Sure, you could hack things up and allow a specific version and build of 
openssl's EVP hashes to dump their state and restore it for use in another 
process running that same specific version and build of openssl (as would 
likely be the case for multiprocessing use) just as you could for any other 
implementation of a hash function such as the builtin libtomcrypt versions.  
But this is not portable between compilations using different implementations 
of the hash algorithm.  That is not what someone using pickle would ever expect.

Public APIs to access the internal state of hash functions do not exist because 
it is not a common thing for people to do.

hashlib isn't going to support this unless someone contributes a very solid 
patch with tests that handles all of the compatibility issues in a friendly 
maintainable manner.

--
stage:  - resolved
type:  - enhancement

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Gregory P. Smith]

Gregory P. Smith g...@krypto.org added the comment:

heh yeah.  while all hash functions do have internal state and someone
could conceivably want to store such a state (it basically amounts to
queued up partial block of input data if any and the current starting
IV) there are not consistent APIs to expose that and I really don't
see why it'd be worth trying to find them.

remember, hashlib doesn't have to be openssl.  there are non openssl
libtomcrypt based versions and someone nice should write a libnss
based version someday.

i'd mark this won't fix. :)

-Greg

On Tue, Apr 5, 2011 at 7:02 AM, Antoine Pitrou rep...@bugs.python.org wrote:

 Antoine Pitrou pit...@free.fr added the comment:

 Why on Earth would you want to serialize a hashlib object?
 It makes as much sense as serializing, say, a JSONEncoder.

 --
 nosy: +gregory.p.smith, pitrou

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue11771
 ___


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Raymond Hettinger]

Raymond Hettinger raymond.hettin...@gmail.com added the comment:

I also recommend closing this one.

--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[STINNER Victor]

Changes by STINNER Victor victor.stin...@haypocalc.com:


--
resolution:  - wont fix
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[STINNER Victor]

New submission from STINNER Victor victor.stin...@haypocalc.com:

$ ./python 
Python 3.3a0 (default:76ed6a061ebe, Apr  5 2011, 12:25:00) 
 import hashlib, pickle
 hash=hashlib.new('md5')
 pickle.dumps(hash)
Traceback (most recent call last):
  File stdin, line 1, in module
_pickle.PicklingError: Can't pickle class '_hashlib.HASH': attribute lookup 
_hashlib.HASH failed

The problem is that _hashlib.HASH is not accessible at Python level. There is a 
C define to make it accessible, but it is disabled by default: #if 
HASH_OBJ_CONSTRUCTOR. This test is as old as the _hashlib module (#1121611, 
624918e1c1b2).

--
components: Library (Lib)
messages: 133021
nosy: haypo
priority: normal
severity: normal
status: open
title: hashlib object cannot be pickled
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

Oh, I don't know if it is possible to serialize a OpenSSL hash object 
(EVP_MD_CTX)...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11771] hashlib object cannot be pickled

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Why on Earth would you want to serialize a hashlib object?
It makes as much sense as serializing, say, a JSONEncoder.

--
nosy: +gregory.p.smith, pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11771
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com