Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Georg Brandl]

Am 03.11.2010 03:35, schrieb Antoine Pitrou:
> On Tue, 2 Nov 2010 19:57:48 -0700
> Brett Cannon  wrote:
>> >
>> > How could we have split the module into a package in a way that matched the
>> > API, whilst still retaining backwards compatibility with the old API? We 
>> > had
>> > no choice but to export the public names at the top level.
>> 
>> I'm not disagreeing with that. What I am saying is can now document
>> that it's unittest.case.TestCase instead of having that just be an
>> implementation detail.
> 
> -1.  unittest.TestCase is far simpler and more obvious that any
> javaesque qualified name.  urllib.request and friends are already
> annoying enough.

Agreed.  There are about 30 names importable from unittest, that is quite
manageable in a single namespace.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

[Victor Stinner]

Le mardi 02 novembre 2010 23:38:12, vous avez écrit :
> On Tue, Nov 2, 2010 at 10:55 PM, Victor Stinner
> 
>  wrote:
> > I don't know how to ignore the BytesWarning without importing warning.
> > How can I do that?
> 
> I was suggesting trying to fix the bootstrap issue so you could use a
> top-level import, instead of working around it with a function level
> import (which we've learned from experience is a recipe for later
> reports from users of programs deadlocking on the import lock - we've
> made lots of improvement to avoid such deadlocks, but still prefer to
> avoid function level imports anyway).

I don't know if there is a bootstrap issue. I'm using a local import because 
os is always loaded at startup, and get_exec_path() is only used to run a 
subprocess: os.exec*() and subprocess.Popen() (only the POSIX implementation). 
I suppose that a top level "import warnings" would augment the memory 
footprint.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 02:57, Brett Cannon wrote:

On Tue, Nov 2, 2010 at 19:50, Michael Foord  wrote:

On 02/11/2010 02:35, Brett Cannon wrote:

On Wed, Oct 27, 2010 at 03:42, Antoine Pitrouwrote:

On Tue, 26 Oct 2010 22:06:37 -0400
Alexander Belopolskywrote:

While I appreciate your and Michael's eloquence, I don't really see
why five 400-line modules are necessarily easier to maintain than one
2000-line module.  Splitting code into modules is certainly a good
thing when the resulting modules can be used independently.  This
helps users write leaner programs, reduces mental footprint of
individual modules, etc, etc.   The split unittest module does not
bring any such benefits.  It still presents a single "big-ball-of-mud"
namespace, only rather than implemented in a single file, it is now
swept in from eight different files.

Are you saying that it has become a pile of medium-sized balls of mud?
I would like to say thanks for the mud, Michael! It's high quality mud
for sure.

I realize I am a little late in this reply but issue 10273 linked to
this and so now I am actually bothering to read this thread since it
felt like bikeshedding when the thread began.

I think the issue here is that the file structure of the code no
longer matches the public API documented by unittest. Personally I,
like most people it seems, prefer source files to be structured in a
way to match the public API. In the case of unittest Michael didn't.

Well the structure *does* match the API (which is primarily why I disagree
with Raymond that this is a 'bad split').

But not the public API as documented, e.g., it's documented as
unittest.TestCase, not unittest.case.TestCase as the file structure
suggests.


Right. I don't think that whether or not the unittest package structure 
is a good structure or not is determined by where we make users import 
the names from. Like others I see little value in reccommending people 
use the longer form for imports.


All the best,

Michael Foord


How could we have split the module into a package in a way that matched the
API, whilst still retaining backwards compatibility with the old API? We had
no choice but to export the public names at the top level.

I'm not disagreeing with that. What I am saying is can now document
that it's unittest.case.TestCase instead of having that just be an
implementation detail.

-Brett


He did ask python-dev if it was okay to do what he did, we all kept
quiet, and now we have realized that most of us prefer to have files

Most of us? Raymond, Alexander and Martin are the only ones I *recall*
complaining about the split specifically in this thread and not all of those
were on the grounds you mention. Several people supported the split. Guido
didn't object to it on these grounds and Antoine noted that finding core
classes was generally straightforward.


[snip...]
So basically it seems like we have learned a lesson: we prefer to have
our code structured in files that match the public API.

When designing packages from the ground up that is a sensible rule of thumb
to follow, but usually follows naturally from good design. This doesn't
necessarily mean that all the sub-modules will export public APIs for
consumers, so this is where I disagree with this. Python's package system is
a very useful way of providing internal structure for projects. That doesn't
mean that this structure must always be exposed publicly.

It should be as easy to navigate as possible (and there is plenty about the
old unittest.py module that wasn't easy to navigate I can assure you), but I
*don't* think that the new package fails in a substantially greater way on
that score.

As Guido points out, this may depend a lot on which tools you use. I could
write more about the role and value of packages, I guess I'll save it for a
blog post.

All the best,

Michael Foord


I think that
is a legitimate design rule for the stdlib to follow from now on, but
in the case of unittest it's too late to change it back (and it's a
minor price to pay to learn this lesson and to have Michael
maintaining unittest like he has been, plus we could consider using
the new structure so that the public API matches the file structure
when the need arises).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileg

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

[Benjamin Peterson]

2010/11/3 Victor Stinner :
> Le mardi 02 novembre 2010 23:38:12, vous avez écrit :
>> On Tue, Nov 2, 2010 at 10:55 PM, Victor Stinner
>>
>>  wrote:
>> > I don't know how to ignore the BytesWarning without importing warning.
>> > How can I do that?
>>
>> I was suggesting trying to fix the bootstrap issue so you could use a
>> top-level import, instead of working around it with a function level
>> import (which we've learned from experience is a recipe for later
>> reports from users of programs deadlocking on the import lock - we've
>> made lots of improvement to avoid such deadlocks, but still prefer to
>> avoid function level imports anyway).
>
> I don't know if there is a bootstrap issue. I'm using a local import because
> os is always loaded at startup, and get_exec_path() is only used to run a
> subprocess: os.exec*() and subprocess.Popen() (only the POSIX implementation).
> I suppose that a top level "import warnings" would augment the memory
> footprint.

Warnings is loaded every time anyway.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Hrvoje Niksic]

On 11/03/2010 01:47 AM, Ben Finney wrote:

 If someone wants to depend on some undocumented detail of the
 directory layout it's their problem (like people depending on bytecode
 and other stuff).


I would say that names without a single leading underscore are part of
the public API, whether documented or not.


I understand this reasoning, but I'd like to offer counter-examples. 
For instance, would you say that glob.glob0 and glob.glob1 are public 
API?  They're undocumented, they're not in __all__, but they don't have 
a leading underscore either, and source comments call them "helper 
functions."  I'm sure there is a lot of other examples like that, both 
in the standard library and in python packages out there.


Other than the existing practice, there is the matter of esthetics. 
Accepting underscore-less identifiers as automatically public leads to a 
proliferation of identifiers with leading underscores, which many people 
(myself included) plainly don't like.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

[Nick Coghlan]

On Wed, Nov 3, 2010 at 10:19 PM, Benjamin Peterson  wrote:
>
> Warnings is loaded every time anyway.

I would have agreed with you, but the contents of sys.modules in a
just-started interactive interpreter suggests that isn't true any more
(which surprised me).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Nick Coghlan]

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
 wrote:
> Sounds like a decision to split a module into a package is a big commitment.  
> Each of the individual file names becomes a permanent part of the API.  Even 
> future additional splits are precluded because it might break someones dotted 
> import (i.e. not a single function can be moved between those files -- once 
> in unittest.utils, alway in unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
  wrote:

Sounds like a decision to split a module into a package is a big commitment.  
Each of the individual file names becomes a permanent part of the API.  Even 
future additional splits are precluded because it might break someones dotted 
import (i.e. not a single function can be moved between those files -- once in 
unittest.utils, alway in unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).


Yes, since unittest.TestCase is still available (as are all the names). 
I believe so anyway...


Michael


Cheers,
Nick.




--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Antoine Pitrou]

Le mercredi 03 novembre 2010 à 14:16 +, Michael Foord a écrit :
> On 03/11/2010 14:05, Nick Coghlan wrote:
> > On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
> >   wrote:
> >> Sounds like a decision to split a module into a package is a big 
> >> commitment.  Each of the individual file names becomes a permanent part of 
> >> the API.  Even future additional splits are precluded because it might 
> >> break someones dotted import (i.e. not a single function can be moved 
> >> between those files -- once in unittest.utils, alway in unittest.utils).
> > Can Python 2.7 pickles containing unittest classes be unpickled using
> > 2.6 or earlier? Even if nobody uses the new names for imports, I
> > believe they implicitly end up included in any pickles involving
> > affected classes (I seem to recall we've been bitten by that before
> > when moving things around).
> 
> Yes, since unittest.TestCase is still available (as are all the names). 
> I believe so anyway...

unittest.TestCase is not really pickleable. There were
test_multiprocessing issues because of that (see recent SVN checkins).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 14:17, Antoine Pitrou wrote:

Le mercredi 03 novembre 2010 à 14:16 +, Michael Foord a écrit :

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
   wrote:

Sounds like a decision to split a module into a package is a big commitment.  
Each of the individual file names becomes a permanent part of the API.  Even 
future additional splits are precluded because it might break someones dotted 
import (i.e. not a single function can be moved between those files -- once in 
unittest.utils, alway in unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).

Yes, since unittest.TestCase is still available (as are all the names).
I believe so anyway...

unittest.TestCase is not really pickleable. There were
test_multiprocessing issues because of that (see recent SVN checkins).


Interesting. We made some fixes before 2.7 to ensure they were copyable, 
but we fixed this in the copy module. TestCase instances now store some 
method objects in a dictionary which may make them unpickleable, so that 
could be a new problem. I'll test with 2.6 and 2.7 to see.


An easy fix would be to store the method names rather than the method 
objects themself (if this is indeed the cause of the problem). This is 
what unittest2 does so that it works with earlier versions of Python 
that don't have the fix we put in copy.


Michael


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Antoine Pitrou]

Le mercredi 03 novembre 2010 à 14:26 +, Michael Foord a écrit :
> 
> Interesting. We made some fixes before 2.7 to ensure they were copyable, 
> but we fixed this in the copy module. TestCase instances now store some 
> method objects in a dictionary which may make them unpickleable, so that 
> could be a new problem. I'll test with 2.6 and 2.7 to see.

I don't think it is a problem in unittest, unless pickling TestCase
objects is really useful. I have fixed the problem in
test_multiprocessing instead.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 14:26, Michael Foord wrote:

On 03/11/2010 14:17, Antoine Pitrou wrote:

Le mercredi 03 novembre 2010 à 14:16 +, Michael Foord a écrit :

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
   wrote:
Sounds like a decision to split a module into a package is a big 
commitment.  Each of the individual file names becomes a permanent 
part of the API.  Even future additional splits are precluded 
because it might break someones dotted import (i.e. not a single 
function can be moved between those files -- once in 
unittest.utils, alway in unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).

Yes, since unittest.TestCase is still available (as are all the names).
I believe so anyway...

unittest.TestCase is not really pickleable. There were
test_multiprocessing issues because of that (see recent SVN checkins).


Interesting. We made some fixes before 2.7 to ensure they were 
copyable, but we fixed this in the copy module. TestCase instances now 
store some method objects in a dictionary which may make them 
unpickleable, so that could be a new problem. I'll test with 2.6 and 
2.7 to see.


An easy fix would be to store the method names rather than the method 
objects themself (if this is indeed the cause of the problem). This is 
what unittest2 does so that it works with earlier versions of Python 
that don't have the fix we put in copy.


Yep, looks like 2.7 introduced a bug making it impossible to pickle 
TestCase instances. I think it will be easy to fix, I'll create a 
specific issue:


$python
Python 2.6.5 (r265:79359, Mar 24 2010, 01:32:55)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from unittest import TestCase
>>> from pickle import dumps
>>> t = TestCase('assert_')
>>> dumps(t)
"ccopy_reg\n_reconstructor\np0\n(cunittest\nTestCase\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'_testMethodDoc'\np6\nS'Fail 
the test unless the expression is 
true.'\np7\nsS'_testMethodName'\np8\nS'assert_'\np9\nsb."

>>>
bigmac:beta.python.org michael$ python2.7
ActivePython 2.7.0.1 (ActiveState Software Inc.) based on
Python 2.7 (r27:82500, Jul  4 2010, 13:58:56)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from unittest import TestCase
>>> from pickle import dumps
>>> t = TestCase('assert_')
>>> dumps(t)
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", 
line 1374, in dumps

Pickler(file, protocol).dump(obj)
...
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", 
line 306, in save

rv = reduce(self.proto)
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", 
line 70, in _reduce_ex

raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

All the best,

Michael


Michael


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk






--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Eric Smith]

On 11/3/10 10:16 AM, Michael Foord wrote:

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
 wrote:

Sounds like a decision to split a module into a package is a big
commitment. Each of the individual file names becomes a permanent
part of the API. Even future additional splits are precluded because
it might break someones dotted import (i.e. not a single function can
be moved between those files -- once in unittest.utils, alway in
unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).


Yes, since unittest.TestCase is still available (as are all the names).
I believe so anyway...


Actually I think the answer is "no" (assuming you could pickle a 
TestCase). Here's an example with TestLoader:


$ python27
Python 2.7.0+ (release27-maint:85878, Oct 28 2010, 06:40:25)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import unittest
>>> x = unittest.TestLoader()
>>> import pickle
>>> pickle.dumps(x)
'ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.'
>>>

$ python24
Python 2.4.4 (#1, Oct 23 2006, 13:58:00)
[GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> 
pickle.loads('ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.')

Traceback (most recent call last):
  File "", line 1, in ?
  File "/usr/lib/python2.4/pickle.py", line 1394, in loads
return Unpickler(file).load()
  File "/usr/lib/python2.4/pickle.py", line 872, in load
dispatch[key](self)
  File "/usr/lib/python2.4/pickle.py", line 1104, in load_global
klass = self.find_class(module, name)
  File "/usr/lib/python2.4/pickle.py", line 1138, in find_class
__import__(module)
ImportError: No module named loader

The problem is that there is no unittest.loader in 2.4, and 
unittest.loader.TestLoader is the name that the 2.7 pickle creates. We 
see this problem every time we try and move anything in the stdlib.


--
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages

[Barry Warsaw]

On Nov 03, 2010, at 11:06 AM, Ben Finney wrote:

>Is this a case where it would be better if the package names had the
>leading underscore: ‘_utils’, ‘_suite’, etc.?
>
>Does the convention on single-leading-underscore identifiers as “don't
>rely on this name staying the same in future versions” hold for package
>names?

I would vote "yes".  I have seen more and more packages use this convention to
signal that the module name is not intended to be imported directly.  This
should be part of any PEP 8 recommendation, IMO.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Barry Warsaw]

On Nov 03, 2010, at 12:34 AM, Antoine Pitrou wrote:

>I don't agree with this. Until it's documented, it's an implementation
>detail and should be able to change without notice.
>If someone wants to depend on some undocumented detail of the directory
>layout it's their problem (like people depending on bytecode and other
>stuff).

+1
-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 14:53, Eric Smith wrote:

On 11/3/10 10:16 AM, Michael Foord wrote:

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
 wrote:

Sounds like a decision to split a module into a package is a big
commitment. Each of the individual file names becomes a permanent
part of the API. Even future additional splits are precluded because
it might break someones dotted import (i.e. not a single function can
be moved between those files -- once in unittest.utils, alway in
unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).


Yes, since unittest.TestCase is still available (as are all the names).
I believe so anyway...


Actually I think the answer is "no" (assuming you could pickle a 
TestCase). Here's an example with TestLoader:




Ah dammit, I read the question the other way round.

Michael


$ python27
Python 2.7.0+ (release27-maint:85878, Oct 28 2010, 06:40:25)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import unittest
>>> x = unittest.TestLoader()
>>> import pickle
>>> pickle.dumps(x)
'ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.' 


>>>

$ python24
Python 2.4.4 (#1, Oct 23 2006, 13:58:00)
[GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> 
pickle.loads('ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.')

Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.4/pickle.py", line 1394, in loads
return Unpickler(file).load()
File "/usr/lib/python2.4/pickle.py", line 872, in load
dispatch[key](self)
File "/usr/lib/python2.4/pickle.py", line 1104, in load_global
klass = self.find_class(module, name)
File "/usr/lib/python2.4/pickle.py", line 1138, in find_class
__import__(module)
ImportError: No module named loader

The problem is that there is no unittest.loader in 2.4, and 
unittest.loader.TestLoader is the name that the 2.7 pickle creates. We 
see this problem every time we try and move anything in the stdlib.





--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

[Benjamin Peterson]

2010/11/3 Nick Coghlan :
> On Wed, Nov 3, 2010 at 10:19 PM, Benjamin Peterson  
> wrote:
>>
>> Warnings is loaded every time anyway.
>
> I would have agreed with you, but the contents of sys.modules in a
> just-started interactive interpreter suggests that isn't true any more
> (which surprised me).

Is that perhaps because of _warnings?



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Eric Smith]

On 11/3/10 10:53 AM, Eric Smith wrote:


The problem is that there is no unittest.loader in 2.4, and
unittest.loader.TestLoader is the name that the 2.7 pickle creates. We
see this problem every time we try and move anything in the stdlib.


And BTW: for me, this is the strongest reason not to break up modules 
into packages or otherwise reorganize the stdlib.


--
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Alexander Belopolsky]

On Tue, Nov 2, 2010 at 6:58 PM, Guido van Rossum  wrote:
..
> To spout a somewhat contrarian opinion, I just browsed the new
> unittest package, and the structure seems reasonable to me, even if
> its submodules are not particularly reusable. I've used this kind of
> style for development myself. What is so offensive about it?

I would not call it "offensive", but what I find annoying is

>>> import unittest
>>> unittest.TestCase.__module__
'unittest.case'

This may not be a problem for smart tools, but for me and a simple
editor what used to be:

Let's find code for unittest.TestCase.

1. Open Lib/unittest.py.
2. Search for "class TestCase".

is now

1. Open Lib/unittest.py
-> No such file or directory.

2. OK, I'm in 2.7.  Open Lib/unittest/__init__.py
3. Search for "class TestCase"
-> beep

4. OK, search for "TestCase"
-> from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf, ..

5. Hmm, what is ".case". Ah, the relative import - open case.py
7.  Search for "class TestCase".
8. What exactly was I looking for?

The above is only slightly exaggerated scenario that I went through
several times when I started using 2.7 before I conditioned myself to
grep in Lib/unittest/*.py.

What is unfortunate is that file split was accompanied by an explosion
of assert* methods in TestCase API which means that anyone reading 2.7
unittests is likely to encounter an unfamiliar method that has to be
looked up.

I think the problem that I described is just a slightly reworded
problem that Raymond reported at the beginning of this thread.   In
other words, I am not alone in seeing this as a problem.

PS: For a "made from scratch" API I would prefer TestCase only be
available from unittest.case.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[James Y Knight]

On Nov 3, 2010, at 11:25 AM, Eric Smith wrote:

> On 11/3/10 10:53 AM, Eric Smith wrote:
> 
>> The problem is that there is no unittest.loader in 2.4, and
>> unittest.loader.TestLoader is the name that the 2.7 pickle creates. We
>> see this problem every time we try and move anything in the stdlib.
> 
> And BTW: for me, this is the strongest reason not to break up modules into 
> packages or otherwise reorganize the stdlib.

This is the strongest reason why I recommend to everyone I know that they not 
use pickle for storage they'd like to keep working after upgrades [not just of 
stdlib, but other 3rd party software or their own software]. :)

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Code coverage doesn't show .py stats

[anatoly techtonik]

Hi,

Python code coverage doesn't include any .py files. What happened?
http://coverage.livinglogic.de/

Did it work before?
--
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Glyph Lefkowitz]

On Nov 3, 2010, at 1:04 PM, James Y Knight wrote:

> This is the strongest reason why I recommend to everyone I know that they not 
> use pickle for storage they'd like to keep working after upgrades [not just 
> of stdlib, but other 3rd party software or their own software]. :)

+1.

Twisted actually tried to preserve pickle compatibility in the bad old days, 
but it was impossible.  Pickles should never really be saved to disk unless 
they contain nothing but lists, ints, strings, and dicts.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 14:53, Eric Smith wrote:

On 11/3/10 10:16 AM, Michael Foord wrote:

On 03/11/2010 14:05, Nick Coghlan wrote:

On Wed, Nov 3, 2010 at 9:32 AM, Raymond Hettinger
 wrote:

Sounds like a decision to split a module into a package is a big
commitment. Each of the individual file names becomes a permanent
part of the API. Even future additional splits are precluded because
it might break someones dotted import (i.e. not a single function can
be moved between those files -- once in unittest.utils, alway in
unittest.utils).

Can Python 2.7 pickles containing unittest classes be unpickled using
2.6 or earlier? Even if nobody uses the new names for imports, I
believe they implicitly end up included in any pickles involving
affected classes (I seem to recall we've been bitten by that before
when moving things around).


Yes, since unittest.TestCase is still available (as are all the names).
I believe so anyway...


Actually I think the answer is "no" (assuming you could pickle a 
TestCase). Here's an example with TestLoader:




It is actually fixable by temporarily switching the __module__ attribute 
of the classes inside a __reduce__ or __reduce_ex__ method. I couldn't 
see a cleaner way of doing it using the pickling protocol methods. I 
asked on #python-dev but the *only* person who claimed to understand the 
pickle protocol methods was Barry, and he is clearly insane.


Antoine is firmly of the opinion that making TestCase instances 
unpickleable is a feature...


Although in practise this is less likely to be an issue for TestCase 
directly as it is extremely rare to use them without subclassing. More 
likely to be an issue for the test result or runner objects.


All the best,

Michael Foord


$ python27
Python 2.7.0+ (release27-maint:85878, Oct 28 2010, 06:40:25)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import unittest
>>> x = unittest.TestLoader()
>>> import pickle
>>> pickle.dumps(x)
'ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.' 


>>>

$ python24
Python 2.4.4 (#1, Oct 23 2006, 13:58:00)
[GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> 
pickle.loads('ccopy_reg\n_reconstructor\np0\n(cunittest.loader\nTestLoader\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.')

Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.4/pickle.py", line 1394, in loads
return Unpickler(file).load()
File "/usr/lib/python2.4/pickle.py", line 872, in load
dispatch[key](self)
File "/usr/lib/python2.4/pickle.py", line 1104, in load_global
klass = self.find_class(module, name)
File "/usr/lib/python2.4/pickle.py", line 1138, in find_class
__import__(module)
ImportError: No module named loader

The problem is that there is no unittest.loader in 2.4, and 
unittest.loader.TestLoader is the name that the 2.7 pickle creates. We 
see this problem every time we try and move anything in the stdlib.





--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Antoine Pitrou]

On Wed, 03 Nov 2010 19:26:53 +
Michael Foord  wrote:
> 
> Antoine is firmly of the opinion that making TestCase instances 
> unpickleable is a feature...

Apparently you didn't really understand me. I'm of the opinion that
making TestCase instances pickleable is useless if that pickling
doesn't have well-defined semantics. And I wonder what the semantics of
pickling a TestCase could be, and what the use cases are.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Jesse Noller]

On Wed, Nov 3, 2010 at 3:45 PM, Antoine Pitrou  wrote:
> On Wed, 03 Nov 2010 19:26:53 +
> Michael Foord  wrote:
>>
>> Antoine is firmly of the opinion that making TestCase instances
>> unpickleable is a feature...
>
> Apparently you didn't really understand me. I'm of the opinion that
> making TestCase instances pickleable is useless if that pickling
> doesn't have well-defined semantics. And I wonder what the semantics of
> pickling a TestCase could be, and what the use cases are.
>
> Regards
>
> Antoine.
>

Splitting groups of tests to run in parallel via multiple processes is
a pretty good use case.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Antoine Pitrou]

Le mercredi 03 novembre 2010 à 15:48 -0400, Jesse Noller a écrit :
> On Wed, Nov 3, 2010 at 3:45 PM, Antoine Pitrou  wrote:
> > On Wed, 03 Nov 2010 19:26:53 +
> > Michael Foord  wrote:
> >>
> >> Antoine is firmly of the opinion that making TestCase instances
> >> unpickleable is a feature...
> >
> > Apparently you didn't really understand me. I'm of the opinion that
> > making TestCase instances pickleable is useless if that pickling
> > doesn't have well-defined semantics. And I wonder what the semantics of
> > pickling a TestCase could be, and what the use cases are.
> >
> > Regards
> >
> > Antoine.
> >
> 
> Splitting groups of tests to run in parallel via multiple processes is
> a pretty good use case.

Indeed, but it implies a lot of things about TestCase instances, which
could have additional non-pickleable attributes (e.g. file objects).
You'd better pickle the TestCase class instead, or simply the module
name as we do with regrtest -jN.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Michael Foord]

On 03/11/2010 19:48, Jesse Noller wrote:

On Wed, Nov 3, 2010 at 3:45 PM, Antoine Pitrou  wrote:

On Wed, 03 Nov 2010 19:26:53 +
Michael Foord  wrote:

Antoine is firmly of the opinion that making TestCase instances
unpickleable is a feature...

Apparently you didn't really understand me. I'm of the opinion that
making TestCase instances pickleable is useless if that pickling
doesn't have well-defined semantics. And I wonder what the semantics of
pickling a TestCase could be, and what the use cases are.

Regards

Antoine.


Splitting groups of tests to run in parallel via multiple processes is
a pretty good use case.


That's something I've been thinking about a lot (and talking to Holger 
about) for the unittest plugins. I definitely won't be doing it with 
pickles but as Antoine says, sending test names to the subprocesses. You 
really want tests run in a child process to behave differently and it 
makes sense to set them up inside the child process.


All the best,

Michael Foord


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Glyph Lefkowitz]

On Nov 3, 2010, at 11:26 AM, Alexander Belopolsky wrote:

> This may not be a problem for smart tools, but for me and a simple
> editor what used to be:


Maybe this is the real problem?  It's 2010, we should all be far enough beyond 
EDLIN that our editors can jump to the definition of a Python class.  Even Vim 
can be convinced to do this ().  
Could Python itself make this easier?  Maybe ship with a command that says 
"hey, somewhere on sys.path, there is a class with .  Please run 
'$EDITOR file +line' (or the current OS's equivalent) so I can look at the 
source code".


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/

[Alexander Belopolsky]

On Wed, Nov 3, 2010 at 4:59 PM, Glyph Lefkowitz  wrote:
..
>  Maybe ship with a command that says "hey, somewhere on sys.path,
> there is a class with .  Please run '$EDITOR file +line' (or the
> current OS's equivalent) so I can look at the source code".
>

Well, we already have inspect.findsource() for that.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

[Nick Coghlan]

On Thu, Nov 4, 2010 at 1:01 AM, Benjamin Peterson  wrote:
> 2010/11/3 Nick Coghlan :
>> On Wed, Nov 3, 2010 at 10:19 PM, Benjamin Peterson  
>> wrote:
>>>
>>> Warnings is loaded every time anyway.
>>
>> I would have agreed with you, but the contents of sys.modules in a
>> just-started interactive interpreter suggests that isn't true any more
>> (which surprised me).
>
> Is that perhaps because of _warnings?

I suspect it's a combination of that and the patch to allow
non-blocking module imports (which turns some things that would
previously have been deadlocks into runtime exceptions).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str.format_from_mapping

[Eric Smith]

On 10/31/10 4:39 PM, Eric Smith wrote:

What are your thoughts on adding a str.format_from_mapping (or similar
name, maybe the suggested "format_map") to 3.2? See
http://bugs.python.org/issue6081 . This method would be similar to
"%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object),
but of course would use str.format syntax: "{foo}
{bar}".format_from_mapping(d).

I like the idea. It's particularly handy when converting from %-formatting.

Eric.


I've updated the issue with tests, minimal docs, and a name change to 
str.format_map. Having heard no objections and some support, I'll commit 
this shortly.


--
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python-3 transition in Arch Linux

[Allan McRae]

Hi,

While this is not strictly related to python development, I thought that 
developers of python might be interested in some of the lessons provided 
by this. So forgive me if this is really wrong for this list...


Recently Arch Linux did a big transition with respect to python.  Now we 
support two python packages: "python" and "python2".


The "python" package will always contain the latest 3.x release and 
currently has /usr/bin/python3.1 with symlinks to /usr/bin/python3 and 
/usr/bin/python.


The "python2" package contains the latest from the "legacy" python-2.x 
branch and has /usr/bin/python2.7 with a symlink to /usr/bin/python2.


I really do not want to debate the sanity of pointing /usr/bin/python at 
python-3.x here, but it suffices to say that I am of the opinion that if 
python-3.x is really the future of python, then /usr/bin/python must 
eventually point to a 3.x version.  Also, Arch Linux is very bleeding 
edge and we expect our users to be competent enough to deal with thing 
like this.  According to #python, we are all idiots  And I have been 
(figuratively) yelled at by a couple of Debian developers (which is 
incidentally the only major distro I found without a /usr/bin/python2 
symlink).


Anyway, this transition was rather simple from a distribution point of 
view apart from the sheer number of packages involved.  All our 
supported packages were rebuilt to work with that symlink layout and any 
"porting" software to use that layout was relatively simple.  Most 
packages either detected the python2 symlink during the rebuild and just 
worked while others required some sort "export PYTHON=python2" or 
"--with-python=python2" or "python2 setup.py" or the like during the build.


Software packages tend to fall into three categories at roughly equal 
frequencies:

1) packages that detected or were pointed at python2 and everything worked
2) packages that detected or were pointed at python2 and partially worked
3) packages that needed adjustment to work with the python2 symlink.

The second case was particularly interesting.  These software would 
change some of their #! to point at the python2 symlink and leave the 
rest pointing at python.  Note that python-2.7 itself falls into this 
category as many files in /usr/lib/python2.7 still have "#!/usr/bin/env 
python" even when installed with "make altinstall".  I can not remember 
the exact details, but I recall that some of these files were installed 
with executable permissions which would be bad, but I need to look into 
this again now things have calmed down...


The packages that did not auto-detect and work with /usr/bin/python2 or 
/usr/bin/python2.7 mostly required a sed of their shebangs or a patch to 
any hardcoded /usr/bin/python paths so were easily fixed.


So that is something that python software developers could think about 
for the future.  When someone configures a module using a particular 
version of python, then ALL shebangs need changed to use that version. 
And it is generally bad practice to hardcode /usr/bin/python into any 
application as you are never quite sure which version you are getting. 
Instead allow it to be configured, keeping the current value as default.


Cheers,
Allan

--
Allan McRae
Arch Linux Developer
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

[anatoly techtonik]

On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz  wrote:
>
> This is the strongest reason why I recommend to everyone I know that they
> not use pickle for storage they'd like to keep working after upgrades [not
> just of stdlib, but other 3rd party software or their own software]. :)
>
> +1.
> Twisted actually tried to preserve pickle compatibility in the bad old days,
> but it was impossible.  Pickles should never really be saved to disk unless
> they contain nothing but lists, ints, strings, and dicts.

But what is alternative in stdlib?
Don't you think that Python doesn't provide any?
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com