Re: [Python-Dev] C.E.R. Thoughts

[Josiah Carlson]

Technical Support of Intercable Co <[EMAIL PROTECTED]> wrote:
> 
> And why not
> if len(sys.argv) > 1 take sys.argv[1] == 'debug':
> ...
> 
> It was not so bad :-)
> 
> A = len(sys.argv)==0 take None or sys.argv[1]
> 
> Sorry for being noisy :-)

The syntax for 2.5 has already been decided upon.  Except for an act by
Guido, it is likely to stay (None if len(sys.argv) == 0 else sys.argv[1]).

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Fredrik Lundh]

Guido van Rossum wrote:

> > > BTW, Queue.Queue violates a recent module naming standard; it is now
> > > considered bad style to name the class and the module the same.
> > > Modules and packages should have short all-lowercase names, classes
> > > should be CapWords. Even the same but different case is bad style.
> >
> > unfortunately, this standard seem to result in generic "spamtools" modules
> > into which people throw everything that's even remotely related to "spam",
> > followed by complaints about bloat and performance from users, followed by
> > various more or less stupid attempts to implement lazy loading of hidden in-
> > ternal modules, followed by more complaints from users who no longer has
> > a clear view of what's really going on in there...
> >
> > I think I'll stick to the old standard for a few more years...
>
> Yeah, until you've learned to use packages. :(

what does packages has to do with this ?  does this new module naming
standard only apply to toplevel package names ?





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Early PEP draft (For Python 3000?)

[Calvin Spealman]

On 10/11/05, Eyal Lotem <[EMAIL PROTECTED]> wrote:
>   locals()['x'] = 1 # Quietly fails!
> Replaced by:
>   frame.x = 1 # Raises error

What about the possibility of making this hypothetic frame object an
indexable, such that frame[0] is the current scope, frame[1] is the
calling scope, etc.? On the same lines, what about closure[0] for the
current frame, while closure[1] resolves to the closure the function
was defined in? These would ensure that you could reliably access any
namespace you would need, without nasty stack tricks and such, and
would make working around some of the limitation of the closures, when
you have such a need. One might even consider a __resolve__ to be
defined in any namespace, allowing all the namespace resolution rules
to be overridden by code at any level.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST branch update

[Guido van Rossum]

[Jeremy]
> > Neil and I have been working on the AST branch for the last week.
> > We're nearly ready to merge the changes to the head.

[Raymond]
> Nice work.

Indeed. I should've threatened to kill the AST branch long ago! :)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extending tuple unpacking

[Nick Coghlan]

Ron Adam wrote:
> I wonder if you make '*' work outside of functions arguments lists, if
> requests to do the same for '**' would follow?

Only if keyword unpacking were to be permitted elsewhere first. That is:

Py> data = dict(a=1, b=2, c=3)
Py> (a, b, c) = **data
Py> print a, b, c
(1, 2, 3)

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extending tuple unpacking

[Nick Coghlan]

Michael Chermside wrote:
> Guido writes:
> 
>>I've always wanted to write that as
>>
>>f(a, b, *args, foo=1, bar=2, **kwds)
>>
>>but the current grammar doesn't allow it.
> 
> 
> Hmm why doesn't the current grammar allow it, and can we fix that?
> I don't see that it's a limitation of the one-token-lookahead, could
> we permit this syntax by rearanging bits of the grammer?

I griped about this a while back, and got the impression from Guido that 
fixing it was possible, but it had simply never bugged anyone enough for them 
to actaully get around to fixing it.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] threadtools (was Re: Autoloading? (Making Queue.Queue easier to use))

[Nick Coghlan]

[EMAIL PROTECTED] wrote:
> Guido> At some level, Queue is just an application of threading, while
> Guido> the threading module provides the basic API ...
> 
> While Queue is built on top of threading Lock and Condition objects, it is a
> highly useful synchronization mechanism in its own right, and is almost
> certainly easier to use correctly (at least for novices) than the
> lower-level synchronization objects the threading module provides.  If
> threading is the "friendly" version of thread, perhaps Queue should be
> considered the "friendly" synchronization object.
> 
> (I'm playing the devil's advocate here.  I'm fine with Queue being where it
> is.)

If we *don't* make Queue a part of the basic threading API (and I think Guido 
is right that it doesn't need to be), then I suggest we create a threadtools 
module.

So the thread-related API would actually have three layers:
   - _thread (currently "_thread") for the low-level guts
   - threading for the basic thread API that any threaded app needs
   - threadtools for the more complex "application-specific" items

Initially threadtools would just contain Queue, but other candidates for 
inclusion in the future might be standard implementations of:
   - PeriodicTimer (see below)
   - FutureCall (threading out a call, only blocking when you need the result
   - QueueThread (a thread with "inbox" and "outbox" Queues)
   - ThreadPool (up to the application to make sure the Threads are reusable)
   - threading related decorators

Cheers,
Nick.

P.S. PeriodicTimer would be a variant of threading Timer which simply replaces 
the run method with:
   def run():
   while 1:
   self.finished.wait(self.interval)
   if self.finished.isSet():
   break
   self.function(*self.args, **self.kwds)


-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Nick Coghlan]

[EMAIL PROTECTED] wrote:
> It sounds like he feels Queue should just be part of threading but queues
> can be used in other contexts besides threading.  So having separate
> modules is a good thing.

If threads aren't involved, you should use "collections.deque" directly, 
rather than going through "Queue.Queue". The latter jumps through a lot of 
hoops in order to be thread-safe.

This confusion is one of the reasons I have a problem with the current name of 
the Queue module.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extending tuple unpacking

[Steve Holden]

Nick Coghlan wrote:
> Ron Adam wrote:
> 
>>I wonder if you make '*' work outside of functions arguments lists, if
>>requests to do the same for '**' would follow?
> 
> 
> Only if keyword unpacking were to be permitted elsewhere first. That is:
> 
> Py> data = dict(a=1, b=2, c=3)
> Py> (a, b, c) = **data
> Py> print a, b, c
> (1, 2, 3)
> 
> Cheers,
> Nick.
> 
This gets too weird, though. What about:

   (a, **d) = **data

Should this be equivalent to

   a = 1
   d = dict(b=2, c=3)

? Basically I suspect we are heading towards the outer limits here.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006  www.python.org/pycon/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Nick Coghlan]

Greg Ewing wrote:
> BTW, I agree that special *syntax* isn't necessarily
> needed. But it does seem to me that some sort of
> hook is needed somewhere to make this doable
> smoothly, that doesn't exist today.

Having module attribute access obey the descriptor protocol (__get__, __set__, 
__delete__) sounds like a pretty good option to me.

It would even be pretty backwards compatible, as I'd be hardpressed to think 
why anyone would have a descriptor *instance* as a top-level object in a 
module (descriptor definition, yes, but not an instance).

Consider lazy instance attributes:

Py> def lazyattr(func):
... class wrapper(object):
... def __get__(self, instance, cls):
... val = func()
... setattr(instance, func.__name__, val)
... return val
... return wrapper()
...
Py> class test(object):
... @lazyattr
... def foo():
... print "Evaluating foo!"
... return "Instance attribute"
...
Py> t = test()
Py> t.foo
Evaluating foo!
'Instance attribute'
Py> t.foo
'Instance attribute'

And lazy class attributes:

Py> def lazyclassattr(func):
... class wrapper(object):
... def __get__(self, instance, cls):
... val = func()
... setattr(cls, func.__name__, val)
... return val
... return wrapper()
...
Py> class test(object):
... @lazyclassattr
... def bar():
... print "Evaluating bar!"
... return "Class attribute"
...
Py> test.bar
Evaluating bar!
'Class attribute'
Py> test.bar
'Class attribute'


Unfortunately, that trick doesn't work at the module level:

Py> def lazymoduleattr(func):
... class wrapper(object):
... def __get__(self, instance, cls):
... val = func()
... globals()[func.__name__] = val
... return val
... return wrapper()
...
Py> @lazymoduleattr
... def baz():
... print "Evaluating baz!"
... return "Module attribute"
...
Py> baz # Descriptor not invoked
<__main__.wrapper object at 0x00B9E3B0>
Py> import sys
Py> main = sys.modules["__main__"]
Py> main.baz # Descriptor STILL not invoked :(
<__main__.wrapper object at 0x00B9E3B0>

But putting the exact same descriptor in a class lets it work its magic:

Py> class lazy(object):
...   baz = baz
...
Py> lazy.baz
Evaluating baz!
'Module attribute'
Py> baz
'Module attribute'

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making Queue.Queue easier to use

[Gustavo J. A. M. Carneiro]

  I'd just like to point out that Queue is not quite as useful as people
seem to think in this thread.  The main problem is that I can't
integrate Queue into a select/poll based main loop.

  The other day I wanted extended a python main loop, which uses poll(),
to be thread safe, so I could queue idle functions from separate
threads.  Obviously Queue doesn't work (no file descriptor to poll), so
I just ended up creating a pipe, to which I send a single byte when I
want to "wake up" the main loop to make it realize changes in its
configuration, such as a new callback added.

  I guess this is partly an unix problem.  There's no system call to say
like "wake me up when one of these descriptors has data OR when this
condition variable is set".  Windows has WaitForMultipleObjects, which I
suspect is quite a bit more powerful.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
The universe is always one step beyond logic.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pythonic concurrency

[Michael Hudson]

Bruce Eckel <[EMAIL PROTECTED]> writes:

> Not only are there significant new library components in
> java.util.concurrent in J2SE5, but perhaps more important is the new
> memory model that deals with issues that are (especially) revealed in
> multiprocessor environments. The new memory model represents new work
> in the computer science field; apparently the original paper is
> written by Ph.D.s and is a bit too theoretical for the normal person
> to follow. But the smart threading guys studied this and came up with
> the new Java memory model -- so that volatile, for example, which
> didn't work quite right before, does now. This is part of J2SE5, and
> this work is being incorporated into the upcoming C++0x.

Do you have a link that explains this sort of thing for the layman?

Cheers,
mwh

-- 
  When physicists speak of a TOE, they don't really mean a theory
  of *everything*.  Taken literally, "Everything" covers a lot of
  ground, including biology, art, decoherence and the best way to
  barbecue ribs.-- John Baez, sci.physics.research
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Michael Hudson]

Greg Ewing <[EMAIL PROTECTED]> writes:

> Phillip J. Eby wrote:
>> At 01:47 PM 10/13/2005 +1300, Greg Ewing wrote:
>> 
>>> I'm trying to change the __class__ of a newly-imported
>>> module to a subclass of types.ModuleType
>> 
>> It happened in Python 2.3, actually.
>
> Is there a discussion anywhere about the reason this was
> done? It would be useful if this capability could be
> regained somehow without breaking things.

Well, I think it's undesirable that you be able to do this to, e.g.,
strings.  Modules are something of a greyer area, I guess.

Cheers,
mwh

-- 
  You sound surprised.  We're talking about a government department
  here - they have procedures, not intelligence.
-- Ben Hutchings, cam.misc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Threading and synchronization primitives

[skip]

Greg> All right then, how about putting it in a module called
Greg> threadutils or something like that, which is clearly related to
Greg> threading, but is open for the addition of future thread-related
Greg> features that might arise.

Then Lock, RLock, Semaphore, etc belong there instead of in threading don't
they?

We have two things here, the basic thread object and the stuff it does (run,
start, etc) and the synchronization primitives.  Thread objects come in two
levels of abstraction: thread.thread and threading.Thread.  The
synchronization primitives come in three levels of abstraction: thread.lock,
threading.{Lock,Semaphore,...} and Queue.Queue.  Each level of abstraction
builds on the level below.

In the typical case I think we want to encourage programmers to use the
highest levels of abstraction available and leave the lower level stuff to
the real pros.  That means most programmers using threads should use
threading.Thread and Queue.Queue.  Partitioning the various classes to
different modules might look like this:

ModuleThread ClassesSync Primitives
-------
_thread   threadlock
threadutils Lock, RLock, Semaphore
threadThreadQueue

Programmers would clearly be discouraged from using the _thread module
(currently thread).  The typical case would be to import the thread module
(currently threading) and use its Thread and Queue objects.  For specialized
use the threadutils programmer can import the threadutils module to get at
the synchronization primitives it contains.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] threadtools (was Re: Autoloading? (Making Queue.Queue easier to use))

[skip]

Nick> So the thread-related API would actually have three layers:
Nick>- _thread (currently "_thread") for the low-level guts
Nick>- threading for the basic thread API that any threaded app needs
Nick>- threadtools for the more complex "application-specific" items

Nick> Initially threadtools would just contain Queue, but other candidates 
for 
Nick> inclusion in the future might be standard implementations of:
Nick>- PeriodicTimer (see below)
Nick>- FutureCall (threading out a call, only blocking when you need 
the result
Nick>- QueueThread (a thread with "inbox" and "outbox" Queues)
Nick>- ThreadPool (up to the application to make sure the Threads are 
reusable)
Nick>- threading related decorators

Given your list of stuff to go in a threadtools module, I still think you
need something to hold Lock, RLock, Condition and Semaphore.  See my
previous post (subject: Threading and synchronization primitives) about a
threadutils module to hold these somewhat lower-level sync primitives.  In
most cases I don't think programmers need them.  OTOH, providing some higher
level abstractions seems to make sense.  (I have to admit I have no idea
what a QueueThread's outbox queue would be used for.  Queues are generally
multi-producer, single-consumer objects.  It makes sense for a thread to
have an inbox.  I'm not so sure about an outbox.)

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] threadtools (was Re: Autoloading? (Making Queue.Queue easier to use))

[Aahz]

On Thu, Oct 13, 2005, [EMAIL PROTECTED] wrote:
> 
> Given your list of stuff to go in a threadtools module, I still think
> you need something to hold Lock, RLock, Condition and Semaphore.  See
> my previous post (subject: Threading and synchronization primitives)
> about a threadutils module to hold these somewhat lower-level sync
> primitives.  In most cases I don't think programmers need them.  OTOH,
> providing some higher level abstractions seems to make sense.  (I
> have to admit I have no idea what a QueueThread's outbox queue would
> be used for.  Queues are generally multi-producer, single-consumer
> objects.  It makes sense for a thread to have an inbox.  I'm not so
> sure about an outbox.)

If you look at my thread tutorial, the spider thread pool uses a
single-producer, multiple-consumer queue to feed URLs to the retrieving
threads.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pythonic concurrency

[Bruce Eckel]

I don't know of anything that exists. There is an upcoming book that
may help:

Java Concurrency in Practice, by Brian Goetz, Tim Peierls, Joshua
Bloch, Joseph Bowbeer, David Holmes, and Doug Lea (Addison-Wesley
2006).

I have had assistance from some of the authors, but don't know if it
introduces the concepts from the research paper. Estimated publication
is February.

However, you might get something from Scott Meyer's analysis of the
concurrency issues surrounding the double-checked locking algorithm:
http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf

Thursday, October 13, 2005, 8:36:21 AM, Michael Hudson wrote:

> Bruce Eckel <[EMAIL PROTECTED]> writes:

>> Not only are there significant new library components in
>> java.util.concurrent in J2SE5, but perhaps more important is the new
>> memory model that deals with issues that are (especially) revealed in
>> multiprocessor environments. The new memory model represents new work
>> in the computer science field; apparently the original paper is
>> written by Ph.D.s and is a bit too theoretical for the normal person
>> to follow. But the smart threading guys studied this and came up with
>> the new Java memory model -- so that volatile, for example, which
>> didn't work quite right before, does now. This is part of J2SE5, and
>> this work is being incorporated into the upcoming C++0x.

> Do you have a link that explains this sort of thing for the layman?

> Cheers,
> mwh



Bruce Eckelhttp://www.BruceEckel.com   mailto:[EMAIL PROTECTED]
Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e"
Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel
Subscribe to my newsletter:
http://www.mindview.net/Newsletter
My schedule can be found at:
http://www.mindview.net/Calendar



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Phillip J. Eby]

At 04:02 PM 10/13/2005 +0100, Michael Hudson wrote:
>Greg Ewing <[EMAIL PROTECTED]> writes:
>
> > Phillip J. Eby wrote:
> >> At 01:47 PM 10/13/2005 +1300, Greg Ewing wrote:
> >>
> >>> I'm trying to change the __class__ of a newly-imported
> >>> module to a subclass of types.ModuleType
> >>
> >> It happened in Python 2.3, actually.
> >
> > Is there a discussion anywhere about the reason this was
> > done? It would be useful if this capability could be
> > regained somehow without breaking things.
>
>Well, I think it's undesirable that you be able to do this to, e.g.,
>strings.  Modules are something of a greyer area, I guess.

Actually, it's desirable to be *able* to do it for anything.  But certainly 
for otherwise-immutable objects it can lead to aliasing issues.

For mutable objects, it's *very* desirable, and I think the rules added in 
2.3 might have been overly strict, as they disallow you changing any 
built-in type to a non built-in type, even if the allocator is the 
same.  It seems to me the safety check could perhaps be reduced to just 
checking whether the old and new classes have the same tp_free.  (Apart 
from the layout and other inheritance-related checks, I mean.)

(By the way, for an example use case other than modules, note that somebody 
wrote an "observables" package that could detect mutation of lists and 
dictionaries in Python 2.2 using __class__ changes, which then became 
useless as of Python 2.3.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Eyal Lotem]

Why not lazily import modules by importing them when they are needed
(i.e inside functions), and not in the top-level module scope?


On 10/13/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> At 04:02 PM 10/13/2005 +0100, Michael Hudson wrote:
> >Greg Ewing <[EMAIL PROTECTED]> writes:
> >
> > > Phillip J. Eby wrote:
> > >> At 01:47 PM 10/13/2005 +1300, Greg Ewing wrote:
> > >>
> > >>> I'm trying to change the __class__ of a newly-imported
> > >>> module to a subclass of types.ModuleType
> > >>
> > >> It happened in Python 2.3, actually.
> > >
> > > Is there a discussion anywhere about the reason this was
> > > done? It would be useful if this capability could be
> > > regained somehow without breaking things.
> >
> >Well, I think it's undesirable that you be able to do this to, e.g.,
> >strings.  Modules are something of a greyer area, I guess.
>
> Actually, it's desirable to be *able* to do it for anything.  But certainly
>
> for otherwise-immutable objects it can lead to aliasing issues.
>
> For mutable objects, it's *very* desirable, and I think the rules added in
> 2.3 might have been overly strict, as they disallow you changing any
> built-in type to a non built-in type, even if the allocator is the
> same.  It seems to me the safety check could perhaps be reduced to just
> checking whether the old and new classes have the same tp_free.  (Apart
> from the layout and other inheritance-related checks, I mean.)
>
> (By the way, for an example use case other than modules, note that somebody
>
> wrote an "observables" package that could detect mutation of lists and
> dictionaries in Python 2.2 using __class__ changes, which then became
> useless as of Python 2.3.)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/eyal.lotem%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Josiah Carlson]

Eyal Lotem <[EMAIL PROTECTED]> wrote:
> Why not lazily import modules by importing them when they are needed
> (i.e inside functions), and not in the top-level module scope?

Because then it wouldn't be automatic.

The earlier portion of this discussion came from...

import module
#module.foo does not reference a module
module.foo
#now module.foo references a module

The discussion is about how we can get that kind of behavior.

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Fredrik Lundh]

Guido van Rossum wrote:

> BTW, Queue.Queue violates a recent module naming standard; it is now
> considered bad style to name the class and the module the same.
> Modules and packages should have short all-lowercase names, classes
> should be CapWords. Even the same but different case is bad style.

unfortunately, this standard seem to result in generic "spamtools" modules
into which people throw everything that's even remotely related to "spam",
followed by complaints about bloat and performance from users, followed by
various more or less stupid attempts to implement lazy loading of hidden in-
ternal modules, followed by more complaints from users who no longer has
a clear view of what's really going on in there...

I think I'll stick to the old standard for a few more years...





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Guido van Rossum]

On 10/13/05, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Guido van Rossum wrote:
>
> > BTW, Queue.Queue violates a recent module naming standard; it is now
> > considered bad style to name the class and the module the same.
> > Modules and packages should have short all-lowercase names, classes
> > should be CapWords. Even the same but different case is bad style.
>
> unfortunately, this standard seem to result in generic "spamtools" modules
> into which people throw everything that's even remotely related to "spam",
> followed by complaints about bloat and performance from users, followed by
> various more or less stupid attempts to implement lazy loading of hidden in-
> ternal modules, followed by more complaints from users who no longer has
> a clear view of what's really going on in there...
>
> I think I'll stick to the old standard for a few more years...

Yeah, until you've learned to use packages. :(

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Antoine Pitrou]

> unfortunately, this standard seem to result in generic "spamtools" modules
> into which people throw everything that's even remotely related to "spam",
> followed by complaints about bloat and performance from users, followed by
> various more or less stupid attempts to implement lazy loading of hidden in-
> ternal modules, followed by more complaints from users who no longer has
> a clear view of what's really going on in there...

BTW, what's the performance problem in importing unnecessary stuff
(assuming pyc files are already generated) ?
Has it been evaluated somewhere ?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Threading and synchronization primitives

[Guido van Rossum]

On 10/13/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
> Greg> All right then, how about putting it in a module called
> Greg> threadutils or something like that, which is clearly related to
> Greg> threading, but is open for the addition of future thread-related
> Greg> features that might arise.
>
> Then Lock, RLock, Semaphore, etc belong there instead of in threading don't
> they?

No. Locks and semaphores are the lowest-level threading primitives.
They go in the basic module.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making Queue.Queue easier to use

[Guido van Rossum]

On 10/13/05, Gustavo J. A. M. Carneiro <[EMAIL PROTECTED]> wrote:
>   I'd just like to point out that Queue is not quite as useful as people
> seem to think in this thread.  The main problem is that I can't
> integrate Queue into a select/poll based main loop.

Well, you're mixing two incompatible paradigms there, so that's to be
expected, right? Either you're using async I/O or you're using
threads. Mixing the two causes confusion and bugs no matter what you
try.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] AST branch update

[Jeremy Hylton]

Neil and I have been working on the AST branch for the last week. 
We're nearly ready to merge the changes to the head.  I imagine we'll
do it this weekend, barring last minute glitches.

There are a few open issues that remain.  I'd like to merge the branch
before resolving them.  Please let me know if you disagree.

The current status of the AST branch is that only two tests fail:
test_trace and test_symtable.  The causes for these failures are
described below.  We did not merge the current head to the branch
again, but I diffed the test suite between head and branch and did not
see any substantive changes since the last merge.

Some of the finer points of generating the line number table (lnotab)
are wrong.  There is some very delicate code to support single
stepping with the debugger.  We'll get that fixed soon, but we'd like
to temporarily disable the failing tests in test_trace.

The symtable module exposed parts of the internal representation of
the old symbol table.  The representation changed, and the module is
going to need to change.  The old module was poorly documented and
tested, so I'd like to start over.  Again, I'd like to disable a
couple of failing tests until after the merge occurs.

I don't think the current test suite covers all of the possible syntax
errors that can be raised.  I'd like to add a new test suite that
covers all of the remaining cases, perhaps moving some existing tests
into this module as well.  I'd like to do that after the merge, which
means there may be some loose ends where syntax errors aren't handled
gracefully.

For those of you familiar with the ast work, I'll summarize the recent changes:

We added line numbers to expressions in the AST.  There are many cases
where a statement spans multiple lines.  We couldn't generate a
correct lnotab without knowing the lines that expressions occur on.

We merged the peephole optimizer into the new compiler and restored
PyNode_Compile() so that the parser module works again.  The parser
module will still expose the old parse trees (just what it's users
want).  We should probably provide a similar AST module, but I'm not
sure if we'll get to that.

We fixed some flawed logic in the symbol table for handling nested
scopes.  Luckily, the test cases for nested scopes are pretty
thorough.  They all pass now.

Jeremy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST branch update

[jepler]

I'm excited to see work continuing (resuming?) on the AST tree.

I don't know how many machines you've been able to test the AST branch on.  I
have a linux/amd64 machine handy and I've tried to run the test suite with a
fresh copy of the ast-branch.

test_trace segfaults consistently, even when run alone.   You didn't give me the
impression that the failure was a segfault, so I'll include more information
about it below.

With '-x test_trace -x test_codecencodings_kr', I get through the testsuite run.
Compared to a build of HEAD, also from today, I get additional failures in
test_genexps test_grp test_pwd test_symtable
and additional unexpected skips of:
test_email test_email_codecs

The 'pwd' and 'grp' failures look like they're due to a change not merged from
HEAD.

I'm not sure what to make of the 'genexps' failure.  Is it just a harmless
output difference?  I didn't see you mention that in your message.

Here is some of the relevant-looking output:
$ ./python -E -tt ./Lib/test/regrtest.py
[...]
**
File "/usr/src/python-ast/Lib/test/test_genexps.py", line ?, in 
test.test_genexps.__test__.doctests
Failed example:
(y for y in (1,2)) = 10
Expected:
Traceback (most recent call last):
   ...
SyntaxError: assign to generator expression not possible
Got:
Traceback (most recent call last):
  File "/usr/src/python-ast/Lib/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
  File "", line 1
 SyntaxError: assignment to generator expression not possible (, line 1)
**
File "/usr/src/python-ast/Lib/test/test_genexps.py", line ?, in 
test.test_genexps.__test__.doctests
Failed example:
(y for y in (1,2)) += 10
Expected:
Traceback (most recent call last):
   ...
SyntaxError: augmented assign to tuple literal or generator expression 
not possible
Got:
Traceback (most recent call last):
  File "/usr/src/python-ast/Lib/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
  File "", line 1
 SyntaxError: augmented assignment to generator expression not possible 
(, line 1)
**
[...]
test test_grp failed -- Traceback (most recent call last):
  File "/usr/src/python-ast/Lib/test/test_grp.py", line 29, in test_values
e2 = grp.getgrgid(e.gr_gid)
OverflowError: signed integer is greater than maximum
[...]
test test_pwd failed -- Traceback (most recent call last):
  File "/usr/src/python-ast/Lib/test/test_pwd.py", line 42, in test_values
self.assert_(pwd.getpwuid(e.pw_uid) in entriesbyuid[e.pw_uid])
OverflowError: signed integer is greater than maximum

The segfault in test_trace looks like this:
$ gdb ./python
(gdb) source Misc/gdbinit
(gdb) run Lib/test/test_trace.py
[...]
test_10_no_jump_to_except_1 (__main__.JumpTestCase) ... FAIL

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 46912496260768 (LWP 11945)]
PyEval_EvalFrame (f=0x652c30) at Python/ceval.c:1994
[1967   case COMPARE_OP:)
1994Py_DECREF(v);
(gdb) print oparg
$1 = 10 [PyCmp_EXC_MATCH?]
(gdb) pyo v
NULL
$2 = void
#0  PyEval_EvalFrame (f=0x652c30) at Python/ceval.c:1994
#1  0x00475800 in PyEval_EvalFrame (f=0x697390) at 
Python/ceval.c:3618
#2  0x00475800 in PyEval_EvalFrame (f=0x694f10) at 
Python/ceval.c:3618
#3  0x00475800 in PyEval_EvalFrame (f=0x649fa0) at 
Python/ceval.c:3618
[...]
#50 0x004113bb in Py_Main (argc=Variable "argc" is not available.) 
at Modules/main.c:484
(gdb) pystack
Lib/test/test_trace.py (447): no_jump_to_except_2
Lib/test/test_trace.py (447): run_test
Lib/test/test_trace.py (557): test_11_no_jump_to_except_2
/usr/src/python-ast/Lib/unittest.py (581): run
/usr/src/python-ast/Lib/unittest.py (280): __call__
/usr/src/python-ast/Lib/unittest.py (420): run
/usr/src/python-ast/Lib/unittest.py (427): __call__
/usr/src/python-ast/Lib/unittest.py (420): run
/usr/src/python-ast/Lib/unittest.py (427): __call__
/usr/src/python-ast/Lib/unittest.py (692): run
/usr/src/python-ast/Lib/test/test_support.py (692): run_suite
/usr/src/python-ast/Lib/test/test_support.py (278): run_unittest
Lib/test/test_trace.py (600): test_main
Lib/test/test_trace.py (600): 
I'm not sure what other information from gdb to furnish.

Jeff


pgpwR8FC2UMjQ.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubs

Re: [Python-Dev] AST branch update

[Neil Schemenauer]

On Thu, Oct 13, 2005 at 05:08:41PM -0500, [EMAIL PROTECTED] wrote:
> test_trace segfaults consistently, even when run alone.

That's a bug in frame_setlineno(), IMO.  It's failing to detect an
invalid jump because the lnotab generated by the new compiler is
slightly different (DUP_TOP opcode corresponds to a different line).

> I'm not sure what to make of the 'genexps' failure.  Is it just a harmless
> output difference?  I didn't see you mention that in your message.

It's a bug in the traceback.py module, IMO.  See bug 1326077.

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Greg Ewing]

Phillip J. Eby wrote:

> Actually, it's desirable to be *able* to do it for anything.  But certainly 
> for otherwise-immutable objects it can lead to aliasing issues.

Even for immutables, it could be useful to be able to
add behaviour that doesn't mutate anything.

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | A citizen of NewZealandCorp, a   |
Christchurch, New Zealand  | wholly-owned subsidiary of USA Inc.  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Simplify lnotab? (AST branch update)

[Greg Ewing]

Jeremy Hylton wrote:

> Some of the finer points of generating the line number table (lnotab)
> are wrong.  There is some very delicate code to support single
> stepping with the debugger.

With disk and memory sizes being what they are nowadays,
is it still worth making heroic efforts to compress the
lnotab table? How about getting rid of all the delicate
code and replacing it with something much simpler?

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | A citizen of NewZealandCorp, a   |
Christchurch, New Zealand  | wholly-owned subsidiary of USA Inc.  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Greg Ewing]

Josiah Carlson wrote:

> The earlier portion of this discussion came from...
> 
> import module
> #module.foo does not reference a module
> module.foo
> #now module.foo references a module

Or more generally, module.foo now references *something*,
not necessarily a module. (In my use case it's a class.)

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | A citizen of NewZealandCorp, a   |
Christchurch, New Zealand  | wholly-owned subsidiary of USA Inc.  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Simplify lnotab? (AST branch update)

[Phillip J. Eby]

At 01:43 PM 10/14/2005 +1300, Greg Ewing wrote:
>Jeremy Hylton wrote:
>
> > Some of the finer points of generating the line number table (lnotab)
> > are wrong.  There is some very delicate code to support single
> > stepping with the debugger.
>
>With disk and memory sizes being what they are nowadays,
>is it still worth making heroic efforts to compress the
>lnotab table? How about getting rid of all the delicate
>code and replacing it with something much simpler?

+1.  I'd be especially interested in lifting the current requirement that 
line ranges and byte ranges both increase monotonically.  Even better if 
the lines for a particular piece of code don't have to all come from the 
same file.  It'd be nice to be able to do the equivalent of '#line' 
directives for Python code that's generated by other tools, such as parser 
generators and the like.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Simplify lnotab? (AST branch update)

[Greg Ewing]

Phillip J. Eby wrote:

> +1.  I'd be especially interested in lifting the current requirement 
> that line ranges and byte ranges both increase monotonically.  Even 
> better if the lines for a particular piece of code don't have to all 
> come from the same file.

How about an array of:

   ++++
   | bytecode index | file no.   |line no.|
   ++++

Entries are sorted by bytecode index, with each entry
applying from that bytecode position up to the position
of the next entry. The file no. indexes a tuple of file
names attached to the code object. All entries are 32-bit
integers.

Easy to generate, easy to look up with a binary search,
should be big enough for everyone except those generating
obscenely huge code objects on 64-bit platforms.

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | A citizen of NewZealandCorp, a   |
Christchurch, New Zealand  | wholly-owned subsidiary of USA Inc.  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))

[Phillip J. Eby]

At 01:32 PM 10/14/2005 +1300, Greg Ewing wrote:
>Phillip J. Eby wrote:
>
> > Actually, it's desirable to be *able* to do it for anything.  But 
> certainly
> > for otherwise-immutable objects it can lead to aliasing issues.
>
>Even for immutables, it could be useful to be able to
>add behaviour that doesn't mutate anything.

I meant that just changing its class is a mutation, and since immutables 
can be shared or cached, that could lead to problems.  So I do think it's a 
reasonable implementation limit to disallow changing the __class__ of an 
immutable.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Simplify lnotab? (AST branch update)

[François Pinard]

[Phillip J. Eby]

> It'd be nice to be able to do the equivalent of '#line' directives for
> Python code that's generated by other tools, such as parser generators
> and the like.

I had such a need a few times in the past, and it was tedious having to
do indirections through generated Python for finding the real source as
referenced by comments.

Yet, granted also that the need has not been frequent, for me.

-- 
François Pinard   http://pinard.progiciels-bpi.ca
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Simplify lnotab? (AST branch update)

[Phillip J. Eby]

At 02:25 PM 10/14/2005 +1300, Greg Ewing wrote:
>Phillip J. Eby wrote:
>
> > +1.  I'd be especially interested in lifting the current requirement
> > that line ranges and byte ranges both increase monotonically.  Even
> > better if the lines for a particular piece of code don't have to all
> > come from the same file.
>
>How about an array of:
>
>++++
>| bytecode index | file no.   |line no.|
>++++
>
>Entries are sorted by bytecode index, with each entry
>applying from that bytecode position up to the position
>of the next entry. The file no. indexes a tuple of file
>names attached to the code object. All entries are 32-bit
>integers.

The file number could be 16-bit - I don't see a use case for referring to 
65,000 different filenames.  ;)  But that doesn't save much space.

Anyway, in the common case, this scheme will use 10 more bytes per line of 
Python code, which translates to a megabyte or so for the standard 
library.  I definitely like the simplicity, but a meg's a meg.  A more 
compact scheme is possible, by using two tables - a bytecode->line number 
table, and a line number-> file table.  In the single-file case, you can 
omit the second table, and the first table then only uses 6 more bytes per 
line than we're currently using.  Not fantastic, but probably more acceptable.

If you have to encode multiple files, you just offset their line numbers by 
the size of the other files, and put entries in the line->file table to 
match.  When computing the line number, you subtract the matching entry in 
the line->file table to get the actual line number within that file.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Early PEP draft (For Python 3000?)

[Greg Ewing]

Eyal Lotem wrote:

>   locals()['x'] = 1 # Quietly fails!
> Replaced by:
>   frame.x = 1 # Raises error

Or even better, replaced by

frame.x = 1 # Does the right thing

The frame object knows enough to be able to find
the correct locals slot and update it, so there's
no need for this to fail.

-- 
Greg Ewing, Computer Science Dept, +--+
University of Canterbury,  | A citizen of NewZealandCorp, a   |
Christchurch, New Zealand  | wholly-owned subsidiary of USA Inc.  |
[EMAIL PROTECTED]  +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 3000 and exec

[Sokolov Yura]

Agree.
 >>>i=1
 >>>def a():
i=2
def b():
print i
return b

 >>>a()()
2
 >>>def a():
i=2
def b():
exec "print i"
return b

 >>>a()()
1

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Pythonic concurrency - offtopic

[Sokolov Yura]

Offtopic:

Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

G:\Working\1>c:\Python24\python
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> from os import fork
Traceback (most recent call last):
  File "", line 1, in ?
ImportError: cannot import name fork
 >>>

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] C.E.R. Thoughts

[Technical Support of Intercable Co]

And why not
if len(sys.argv) > 1 take sys.argv[1] == 'debug':
...

It was not so bad :-)

A = len(sys.argv)==0 take None or sys.argv[1]

Sorry for being noisy :-)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode charmap decoders slow

[Tony Nelson]

I have written my fastcharmap decoder and encoder.  It's not meant to be
better than the patch and other changes to come in a future version of
Python, but it does work now with the current codecs.  Using Hye-Shik
Chang's benchmark, decoding is about 4.3x faster than the base, and
encoding is about 2x faster than the base (that's comparing the base and
the fast versions on my machine).  If fastcharmap would be useful, please
tell me where I should make it available, and any changes that are needed.
I would also need to write an installer (distutils I guess).



Fastcharmap is written in Python and Pyrex 0.9.3, and the .pyx file will
need to be compiled before use.  I used:

pyrexc _fastcharmap.pyx
gcc -c -fPIC -I/usr/include/python2.3/ _fastcharmap.c
gcc -shared _fastcharmap.o -o _fastcharmap.so

To use, hook each codec to be speed up:

import fastcharmap
help(fastcharmap)
fastcharmap.hook('name_of_codec')
u = unicode('some text', 'name_of_codec')
s = u.encode('name_of_codec')

No codecs were rewritten.  It took me a while to learn enough to do this
(Pyrex, more Python, some Python C API), and there were some surprises.
Hooking in is grosser than I would have liked.  I've only used it on Python
2.3 on FC3.  Still, it should work going forward, and, if the dicts are
replaced by something else, fastcharmap will know to leave everything
alone.  There's still a tiny bit of debugging print statements in it.


>At 8:36 AM +0200 10/5/05, Martin v. Löwis wrote:
>>Tony Nelson wrote:
> ...
>>> Encoding can be made fast using a simple hash table with external chaining.
>>> There are max 256 codepoints to encode, and they will normally be well
>>> distributed in their lower 8 bits.  Hash on the low 8 bits (just mask), and
>>> chain to an area with 256 entries.  Modest storage, normally short chains,
>>> therefore fast encoding.
>>
>>This is what is currently done: a hash map with 256 keys. You are
>>complaining about the performance of that algorithm. The issue of
>>external chaining is likely irrelevant: there likely are no collisions,
>>even though Python uses open addressing.
>
>I think I'm complaining about the implementation, though on decode, not
>encode.
>
>In any case, there are likely to be collisions in my scheme.  Over the
>next few days I will try to do it myself, but I will need to learn Pyrex,
>some of the Python C API, and more about Python to do it.
>
>
...I suggest instead just /caching/ the translation in C arrays stored
with the codec object.  The cache would be invalidated on any write to the
codec's mapping dictionary, and rebuilt the next time anything was
translated.  This would maintain the present semantics, work with current
codecs, and still provide the desired speed improvement.
>>
>>That is not implementable. You cannot catch writes to the dictionary.
>
>I should have been more clear.  I am thinking about using a proxy object
>in the codec's 'encoding_map' and 'decoding_map' slots, that will forward
>all the dictionary stuff.  The proxy will delete the cache on any call
>which changes the dictionary contents.  There are proxy classed and
>dictproxy (don't know how its implemented yet) so it seems doable, at
>least as far as I've gotten so far.
>
>
>>> Note that this caching is done by new code added to the existing C
>>> functions (which, if I have it right, are in unicodeobject.c).  No
>>> architectural changes are made; no existing codecs need to be changed;
>>> everything will just work
>>
>>Please try to implement it. You will find that you cannot. I don't
>>see how regenerating/editing the codecs could be avoided.
>
>Will do!

TonyN.:'   
  '  
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Autoloading? (Making Queue.Queue easier to use)

[Sokolov Yura]

May be allow modules to define __getattr__ ?

def __getattr__(thing):
 try:
  return __some_standart_way__(thing)
except AttributeError:
  if thing=="Queue":
   import sys
   from Queue import Queue
   setattr(sys.modules[__name__],"Queue",Queue)
   return Queue
  raise


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST branch update

[Raymond Hettinger]

> Neil and I have been working on the AST branch for the last week.
> We're nearly ready to merge the changes to the head. 

Nice work.



> I don't think the current test suite covers all of the possible syntax
> errors that can be raised. 

Do the AST branch generate a syntax error for:

   foo(a = i for i in range(10))

?



Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pythonic concurrency - offtopic

[Josiah Carlson]

Sokolov Yura <[EMAIL PROTECTED]> wrote:
> 
> Offtopic:
> 
> Microsoft Windows [Version 5.2.3790]
> (C) Copyright 1985-2003 Microsoft Corp.
> 
> G:\Working\1>c:\Python24\python
> Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on 
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> from os import fork
> Traceback (most recent call last):
>   File "", line 1, in ?
> ImportError: cannot import name fork
>  >>>

Python for Windows, if I remember correctly, has never supported forking. 
This is because the underlying process execution code does not have
support for the standard copy-on-write semantic which makes unix fork
fast.

Cygwin Python does support fork, but I believe this is through a literal
copying of the memory space, which is far slower than unix fork.

Until Microsoft adds kernel support for fork, don't expect standard
Windows Python to support it.

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST branch update

[Neil Schemenauer]

On Fri, Oct 14, 2005 at 01:03:28AM -0400, Raymond Hettinger wrote:
> Do the AST branch generate a syntax error for:
> 
>foo(a = i for i in range(10))

No.  It generates the same broken code as the current compiler.

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com