Re: [Python-Dev] test_ctypes failure on Mac OS X/PowerPC 10.3.9 (Panther)

2006-06-21 Thread Thomas Heller
Ronald Oussoren schrieb:
 
 On 20-jun-2006, at 20:50, Ronald Oussoren wrote:
 

 On 20-jun-2006, at 20:06, Thomas Heller wrote:

 Trent Mick schrieb:
 Thomas and others,

 Has anyone else seen failures in test_ctypes on older Mac OS X/
 PowerPC?
 Results are below. This is running a build of the trunk from last
 night:

 ./configure  make  ./python.exe Lib/test/test_ctypes.py

 Note that the test does NOT fail on the Mac OS X/x86 10.4.6 box
 that I have.

 It also works on 10.4.?? Power PC.  I guess the fix has to wait until
 I'm able to install 10.3 on my mac, I have the DVDs already but
 have not
 yet had the time.  If anyone is willing to give me ssh access to a
 10.3
 box I can try to fix this earlier.

 I had some problems with my 10.3-capable box, but happily enough it
 decided to come alive again. I'm currently booted into 10.3.9 and
 will have a look.
 
 It is a platform bug, RTLD_LOCAL doesn't work on 10.3. The following C 
 snippet fails with the same error as ctypes: FAIL: dlcompat: unable to 
 open this file with RTLD_LOCAL. This seems to be confirmed by this 
 sourcet test file from darwin: 
 http://darwinsource.opendarwin.org/10.4.1/dyld-43/unit-tests/test-cases/dlopen-RTLD_LOCAL/main.c.
  
 

What does this mean?  Would it work with RTLD_GLOBAL, is there any other 
way to repair it, or does loading dylibs not work at all on Panther?

Thomas
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] test_ctypes failure on Mac OS X/PowerPC 10.3.9(Panther)

2006-06-21 Thread Ronald Oussoren
 
On Wednesday, June 21, 2006, at 09:43AM, Thomas Heller [EMAIL PROTECTED] 
wrote:

Ronald Oussoren schrieb:
 will have a look.
 
 It is a platform bug, RTLD_LOCAL doesn't work on 10.3. The following C 
 snippet fails with the same error as ctypes: FAIL: dlcompat: unable to 
 open this file with RTLD_LOCAL. This seems to be confirmed by this 
 sourcet test file from darwin: 
 http://darwinsource.opendarwin.org/10.4.1/dyld-43/unit-tests/test-cases/dlopen-RTLD_LOCAL/main.c.
  
 

What does this mean?  Would it work with RTLD_GLOBAL, is there any other 
way to repair it, or does loading dylibs not work at all on Panther?

Using RTLD_GLOBAL does work. This should also be fairly save as RTLD_GLOBAL 
seems to be the same as RTLD_LOCAL when using two-level namespaces (which is 
the default on OSX and used by Python).

Ronald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Ka-Ping Yee
On Wed, 21 Jun 2006, Phillip J. Eby wrote:
 Well, EIBTI and all that:

  switch x:
  case == 1: foo(x)
  case in S: bar(x)

 It even lines up nicely.  :)

Hmm, this is rather nice.  I can imagine possible use cases for

switch x:
case  3: foo(x)
case is y: spam(x)
case == z: eggs(x)

An interesting use case for which this offers no corresponding
syntax is

case instanceof ClassA: ham(x)

which doesn't work because Python spells a type test as
isinstance(a, b) rather than with an operator.  (I suppose
whether we want it to be an operator might be another
question to think about for Python 3000.)


-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Georg Brandl
Ka-Ping Yee wrote:
 On Wed, 21 Jun 2006, Phillip J. Eby wrote:
 Well, EIBTI and all that:

  switch x:
  case == 1: foo(x)
  case in S: bar(x)

 It even lines up nicely.  :)
 
 Hmm, this is rather nice.  I can imagine possible use cases for
 
 switch x:
 case  3: foo(x)
 case is y: spam(x)

Ha, a slight reminiscence of BASIC...

 case == z: eggs(x)
 
 An interesting use case for which this offers no corresponding
 syntax is
 
 case instanceof ClassA: ham(x)
 
 which doesn't work because Python spells a type test as
 isinstance(a, b) rather than with an operator.  (I suppose
 whether we want it to be an operator might be another
 question to think about for Python 3000.)

FWIW, I like is a most, but there's no way to spell this
as one word without confusing readers.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Nick Coghlan
Greg Ewing wrote:
 Phillip J. Eby wrote:
 
 Actually, one could consider case expressions to be computed at function 
 definition time, the way function defaults are.  That would solve the 
 problem of symbolic constants, or indeed any sort of expressions.
 
 That's an excellent idea!
 
 It's just a question of which one is easier to explain. 
 
 I think the function-definition-time one is easiest to
 both explain and also to reason about when writing code,
 since definition time is well-defined, whereas the first
 time it's executed is somewhat fuzzy.

There's some benefit to first time it's executed though:
   a. it allows access to the local namespace
   b. it uses the same semantics at module level as it does in a function

If we go with 'at function definition time', then neither of those is true. 
I'm actually curious how a module level switch statement would work at all in 
that case, without either falling back on the first time it's executed 
definition, or else not permitting switch statements in module level code.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Nick Coghlan
Facundo Batista wrote:
 2006/6/20, Nick Coghlan [EMAIL PROTECTED]:
 The intent was always to replace the internal use of tuples and longs 
 with a
 more efficient C implementation - that particular step simply wasn't 
 needed
 for the original use case that lead Facundo to write and implement PEP 
 327.
 
 Right. We never addressed speed. I mean, we made Decimal as fast as we
 could in the limited time we had (Raymond H. helped a lot also here),
 but it was NOT designed for speed.

As I recall, the design flow was pretty much 'make it work to spec' then 'make 
it run the telco benchmark and the tests faster while still keeping the 
implementation reasonably simple'. Helping Raymond with that tuning process 
was actually my first real contribution to CPython, so I got a lot of reading 
done while waiting for the benchmark and the decimal arithmetic tests to run 
with the Python profiler enabled ;)

Even then, I believe only two particularly significant changes were made to 
the implementation - adding the boolean flag so special values could be 
detected easily, and copping the conversion costs to  from longs for 
coefficient arithmetic, because we made the time back in the long run by 
getting to use the C-coded long arithmetic operations.

 BTW, prove me Decimal is not fast enough, ;)

C:\Python24python -m timeit -s x = 1.0 x+x
1000 loops, best of 3: 0.137 usec per loop

C:\Python24python -m timeit -s from decimal import Decimal as d; x = d(1) 
x+x
1 loops, best of 3: 48.3 usec per loop

I don't really know my definition of 'fast enough to be the basic floating 
point type', but I'm pretty sure that a couple of orders of magnitude slower 
isn't it. I guess I'll find out what my definition is if the C implementation 
manages to get there ;)

(Hmm - a couple of spot checks makes it look like the decimal module's slowed 
down by a few percent in Python 2.5. It's probably worth trying out the new 
profiler on the module to see if there are any simple fixes to be made before 
beta 2. . .)

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Phillip J. Eby
At 03:38 AM 6/21/2006 -0500, Ka-Ping Yee wrote:
On Wed, 21 Jun 2006, Phillip J. Eby wrote:
  Well, EIBTI and all that:
 
   switch x:
   case == 1: foo(x)
   case in S: bar(x)
 
  It even lines up nicely.  :)

Hmm, this is rather nice.  I can imagine possible use cases for

 switch x:
 case  3: foo(x)
 case is y: spam(x)
 case == z: eggs(x)

An interesting use case for which this offers no corresponding
syntax is

 case instanceof ClassA: ham(x)

Actually, I was assuming that any other operator besides == and 'in' would 
be relegated to an if-elif chain in the default case, although it's almost 
possible to do that automatically, I suppose.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Aahz
On Wed, Jun 21, 2006, Nick Coghlan wrote:
 Facundo Batista wrote:
 
 BTW, prove me Decimal is not fast enough, ;)
 
 C:\Python24python -m timeit -s x = 1.0 x+x
 1000 loops, best of 3: 0.137 usec per loop
 
 C:\Python24python -m timeit -s from decimal import Decimal as d; x = d(1) 
 x+x
 1 loops, best of 3: 48.3 usec per loop
 
 I don't really know my definition of 'fast enough to be the basic
 floating point type', but I'm pretty sure that a couple of orders of
 magnitude slower isn't it. I guess I'll find out what my definition is
 if the C implementation manages to get there ;)

Why isn't that fast enough?  Relative speed is *not* the issue when
talking about real-world applications.  More to the point, the
expectation is that the C implementation of Decimal will have faster
conversion to/from string, which in many real world applications forms a
significant part of the processing load.
-- 
Aahz ([EMAIL PROTECTED])   * http://www.pythoncraft.com/

I saw `cout' being shifted Hello world times to the left and stopped
right there.  --Steve Gonedes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Fredrik Lundh
Nick Coghlan wrote:

 BTW, prove me Decimal is not fast enough, ;)
 
 C:\Python24python -m timeit -s x = 1.0 x+x
 1000 loops, best of 3: 0.137 usec per loop
 
 C:\Python24python -m timeit -s from decimal import Decimal as d; x = d(1) 
 x+x
 1 loops, best of 3: 48.3 usec per loop
 
 I don't really know my definition of 'fast enough to be the basic floating 
 point type', but I'm pretty sure that a couple of orders of magnitude slower 
 isn't it.

how fast does the corresponding C program run ?

/F

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Paul Moore
On 6/21/06, Fredrik Lundh [EMAIL PROTECTED] wrote:
 Nick Coghlan wrote:

  BTW, prove me Decimal is not fast enough, ;)
 
  C:\Python24python -m timeit -s x = 1.0 x+x
  1000 loops, best of 3: 0.137 usec per loop
 
  C:\Python24python -m timeit -s from decimal import Decimal as d; x = 
  d(1) x+x
  1 loops, best of 3: 48.3 usec per loop
 
  I don't really know my definition of 'fast enough to be the basic floating
  point type', but I'm pretty sure that a couple of orders of magnitude slower
  isn't it.

 how fast does the corresponding C program run ?

A horribly crude test using Regina Rexx (which implements the Decimal
standard in C, but I know nothing else about how):

x = 1.0
do 1000
  y = x + x
end

This takes about 5 sec on my PC, which (if I've calculated correctly)
comes out at 0.5 usec per loop.

I suppose it gives a ballpark figure for the sorts of speeds we can
expect from a C implementation.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Nick Coghlan [EMAIL PROTECTED] wrote:
 There's some benefit to first time it's executed though:
a. it allows access to the local namespace

And how would that be a good thing? It just begs for confusion if the
local variable doesn't always have the same value. (Yes, globals may
vary too, but less likely, since global *variables* (i.e. that
actually vary) are generally considered a bad idea. There's no such
taboo for local variables. :-)

b. it uses the same semantics at module level as it does in a function

Hm, I hadn't thought of that one yet.

 If we go with 'at function definition time', then neither of those is true.
 I'm actually curious how a module level switch statement would work at all in
 that case, without either falling back on the first time it's executed
 definition, or else not permitting switch statements in module level code.

After thinking about it a bit I think that if it's not immediately
contained in a function, it should be implemented as alternative
syntax for an if/elif chain.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
 At 03:38 AM 6/21/2006 -0500, Ka-Ping Yee wrote:
 On Wed, 21 Jun 2006, Phillip J. Eby wrote:
   Well, EIBTI and all that:
  
switch x:
case == 1: foo(x)
case in S: bar(x)
  
   It even lines up nicely.  :)
 
 Hmm, this is rather nice.  I can imagine possible use cases for
 
  switch x:
  case  3: foo(x)
  case is y: spam(x)
  case == z: eggs(x)
 
 An interesting use case for which this offers no corresponding
 syntax is
 
  case instanceof ClassA: ham(x)

 Actually, I was assuming that any other operator besides == and 'in' would
 be relegated to an if-elif chain in the default case, although it's almost
 possible to do that automatically, I suppose.

I've been thinking about generalization to other operators too, but
decided that it would be a mistake. It would be quite clumsy to
explain the exact semantics: if all operators are == or in an
efficient hash table gets pre-constructed at function definition time,
otherwise, um..., what exactly?

(Note how I've switched to the switch-for-efficiency camp, since it
seems better to have clear semantics and a clear reason for the syntax
to be different from if/elif chains.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] RELEASED Python 2.5 (beta 1)

2006-06-21 Thread Anthony Baxter
On behalf of the Python development team and the Python community, I'm 
happy to announce the first BETA release of Python 2.5.

This is an *beta* release of Python 2.5. As such, it is not suitable 
for a production environment. It is being released to solicit 
feedback and hopefully discover bugs, as well as allowing you to 
determine how changes in 2.5 might impact you. If you find things 
broken or incorrect, please log a bug on Sourceforge. 

I'd like to really encourage you to try out this version and check 
that your code still works - if not, and you think it's a bug, please 
log a bug. Hopefully this will make it easier for you to upgrade once 
the final release of Python 2.5 is done.

Please note that changes to improve Python's support for 64 bit 
systems might require authors of C extensions to change their code. 
See the website for more, including a link to a posting discussing 
this issue in particular.

More information on the release (as well as source distributions and 
Windows and Mac OSX installers) are available from the 2.5 website:

http://www.python.org/2.5/

Since the alpha releases, a slew of bug fixes and smaller new
features have been added. See the release notes (available from the
2.5 webpage) for more. The first beta also includes the results of the 
Iceland NeedForSpeed sprint, resulting in some significant speedups.

As of this release, Python 2.5 is now in *feature freeze*. No new
features are planned - only bugfixes for the code already in the 
codebase.

The plan from here is for one more beta release followed by one or 
more release candidates as needed, leading to a 2.5 final release 
early August.  PEP 356 includes the schedule and will be updated as 
the schedule evolves.

The new features in Python 2.5 are described in Andrew Kuchling's 
What's New In Python 2.5. It's available from the 2.5 web page.

Amongst the language features added include conditional expressions, 
the with statement, the merge of try/except and try/finally into 
try/except/finally, enhancements to generators to produce a coroutine 
kind of functionality, and a brand new AST-based compiler 
implementation.

New modules added include hashlib, ElementTree, sqlite3, wsgiref and
ctypes. We also have a new profiling module cProfile.

Enjoy this new release (another step on the path to Python 2.5 final)
Anthony

-- 
Anthony Baxter
[EMAIL PROTECTED]
Python Release Manager
(on behalf of the entire python-dev team)


pgpxpkNRaGBAW.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Phillip J. Eby
At 09:16 AM 6/21/2006 -0700, Guido van Rossum wrote:
After thinking about it a bit I think that if it's not immediately
contained in a function, it should be implemented as alternative
syntax for an if/elif chain.

That worries me a little.  Suppose I write a one-off script like this:

for line in sys.stdin:
 words = line.split()
 if words:
 switch words[0]:
 case foo: blah
 case words[-1]: print mirror image!

Then, if I later move the switch into a function, it's not going to mean 
the same thing any more.  If the values are frozen at first use or 
definition time (which are the same thing for module-level code), then I'll 
find the lurking bug sooner.

OTOH, breaking it sooner doesn't seem like such a great idea either; seems 
like a recipe for a newbie-FAQ, actually.  ISTM that the only sane way to 
deal with this would be to ban the switch statement at module level, which 
then seems to be an argument for not including the switch statement at all.  :(

I suppose the other possibility would be to require at compilation time 
that a case expression include only non-local variables.  That would mean 
that you couldn't use *any* variables in a case expression at module-level 
switch, but wording the error message for that to not be misleading might 
be tricky.

I suppose an error message for the above could simply point to the fact 
that 'words' is being rebound in the current scope, and thus can't be 
considered a constant.  This is only an error at the top-level if the 
switch appears in a loop, and the variable is rebound somewhere within that 
loop or is rebound more than once in the module as a whole (including 
'global' assignments in functions).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] TRUNK is UNFROZEN, but in FEATURE FREEZE

2006-06-21 Thread Anthony Baxter
2.5b1 is out, so I'm declaring the SVN trunk unfrozen. Note, though, 
that as we're now post-beta, we're in FEATURE FREEZE. 

Really. This means you. :-)

No new features should be checked in without prior approval - checkins 
that violate this will quite probably get backed out.

I expect that we will also now be quite a bit more anal about any 
checkins that break the buildbots. Please, please make sure you run 
the test suite before checking in - and if you're at all concerned 
that your checkin might have strange platform dependencies, check the 
buildbot status page (http://www.python.org/dev/buildbot/trunk/) 
after your checkin to make sure it didn't break anything. Similarly, 
if you're fixing a bug, if at all possible write a test and check 
that in as well. 

The buildbots and a focus on testing should mean that 2.5 ends up 
being one of the most solid Python releases so far. Please help us 
achieve this goal.

The feature freeze on the trunk will continue until we branch for 
release candidate 1 of 2.5 - sometime in the second half of July, 
probably. If you really have the need to do new work on the trunk 
before then, please work on a branch. 

Thanks,
Anthony
-- 
Anthony Baxter [EMAIL PROTECTED]
It's never too late to have a happy childhood.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Fredrik Lundh
Guido van Rossum wrote:

 (Note how I've switched to the switch-for-efficiency camp, since it
 seems better to have clear semantics and a clear reason for the syntax
 to be different from if/elif chains.)

if you're now in the efficiency camp, why not just solve this on the 
code generator level ?  given

 var = some expression
 if var == constant:
 ...
 elif var == constant:
 ...

let the compiler use a dispatch table, if it can and wants to.

/F

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Fredrik Lundh [EMAIL PROTECTED] wrote:
 Guido van Rossum wrote:

  (Note how I've switched to the switch-for-efficiency camp, since it
  seems better to have clear semantics and a clear reason for the syntax
  to be different from if/elif chains.)

 if you're now in the efficiency camp, why not just solve this on the
 code generator level ?  given

  var = some expression
  if var == constant:
  ...
  elif var == constant:
  ...

 let the compiler use a dispatch table, if it can and wants to.

But in most cases the 'constant' is actually an expression involving a
global, often even a global in another module. (E.g. sre_compile.py)
The compiler will have a hard time proving that this is really a
constant, so it won't optimize the code.

The proposed switch semantics (create the table when the containing
function is defined) get around this by defining what it means by
constant.

BTW I would like references to locals shadowing globals to be flagged
as errors (or at least warnings) so that users who deduced the wrong
mental model for a switch statement are caught out sooner.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
 At 09:16 AM 6/21/2006 -0700, Guido van Rossum wrote:
 After thinking about it a bit I think that if it's not immediately
 contained in a function, it should be implemented as alternative
 syntax for an if/elif chain.

 That worries me a little.  Suppose I write a one-off script like this:

 for line in sys.stdin:
  words = line.split()
  if words:
  switch words[0]:
  case foo: blah
  case words[-1]: print mirror image!

Why would you write a script like that? If you've learned the
idiomatic use of a switch statement first, that would never occur to
you. If you're totally clueless, I don't really care that much.

 Then, if I later move the switch into a function, it's not going to mean
 the same thing any more.

And it will be a clear compile-time warning because in the refactored
version you'd be attempting to use a local variable in a case.

 If the values are frozen at first use or
 definition time (which are the same thing for module-level code), then I'll
 find the lurking bug sooner.

Or not, depending on how easily the misbehavior is spotted from a
cursory glance at the output.

 OTOH, breaking it sooner doesn't seem like such a great idea either; seems
 like a recipe for a newbie-FAQ, actually.  ISTM that the only sane way to
 deal with this would be to ban the switch statement at module level, which
 then seems to be an argument for not including the switch statement at all.  
 :(

I don't understand this line of reasoning. The semantics I propose are
totally well-defined.

 I suppose the other possibility would be to require at compilation time
 that a case expression include only non-local variables.  That would mean
 that you couldn't use *any* variables in a case expression at module-level
 switch, but wording the error message for that to not be misleading might
 be tricky.

That seems overly restrictive given that I expect *most* cases to use
named constants, not literals.

 I suppose an error message for the above could simply point to the fact
 that 'words' is being rebound in the current scope, and thus can't be
 considered a constant.  This is only an error at the top-level if the
 switch appears in a loop, and the variable is rebound somewhere within that
 loop or is rebound more than once in the module as a whole (including
 'global' assignments in functions).

Let's not focus on the error message. I think your assumption that
every switch at the global level ought to be able to be moved into a
function and work the same way is not a particularly important
requirement.

(As a compromise, a switch at the global level with only literal cases
could be efficiently optimized. This should include compile-time
constant expressions.)

BTW a switch in a class should be treated the same as a global switch.
But what about a switch in a class in a function?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Phillip J. Eby
At 06:41 PM 6/21/2006 +0200, Fredrik Lundh wrote:
Guido van Rossum wrote:

  (Note how I've switched to the switch-for-efficiency camp, since it
  seems better to have clear semantics and a clear reason for the syntax
  to be different from if/elif chains.)

if you're now in the efficiency camp, why not just solve this on the
code generator level ?  given

  var = some expression
  if var == constant:
  ...
  elif var == constant:
  ...

let the compiler use a dispatch table, if it can and wants to.

Two reasons:

1. Having special syntax is an assertion that 'var' will be usable as a 
dictionary key.  Without this assertion, the generated code would need to 
trap hashing failure.

2. Having special syntax is likewise an assertion that the 'constants' will 
remain constant, if they're symbolic constants like:

FOO = foo


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Phillip J. Eby
At 09:55 AM 6/21/2006 -0700, Guido van Rossum wrote:
BTW a switch in a class should be treated the same as a global switch.
But what about a switch in a class in a function?

Okay, now my head hurts.  :)

A switch in a class doesn't need to be treated the same as a global switch, 
because locals()!=globals() in that case.

I think the top-level is the only thing that really needs a special case 
vs. the general error if you use a local variable in the expression rule.

Actually, it might be simpler just to always reject local variables -- even 
at the top-level -- and be done with it.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
 At 09:55 AM 6/21/2006 -0700, Guido van Rossum wrote:
 BTW a switch in a class should be treated the same as a global switch.
 But what about a switch in a class in a function?

 Okay, now my head hurts.  :)

Welcome to the club. There's a Monty Python sketch appropriate...

 A switch in a class doesn't need to be treated the same as a global switch,
 because locals()!=globals() in that case.

But that's not the discerning rule in my mind; the rule is, how to
define at function definition time.

 I think the top-level is the only thing that really needs a special case
 vs. the general error if you use a local variable in the expression rule.

To the contrary, at the top level my preferred semantics don't care
because they don't use a hash.

The strict rules about locals apply when it occurs inside a function,
since then we eval the case expressions at function definition time,
when the locals are undefined. This would normally be good enough, but
I worry (a bit) about this case:

  y = 12
  def foo(x, y):
switch x:
case y: print something

which to the untrained observer (I care about untrained readers much
more than about untrained writers!) looks like it would print
something if x equals y, the argument, while in fact it prints
something if x equals 12.

 Actually, it might be simpler just to always reject local variables -- even
 at the top-level -- and be done with it.

Can't because locals at the top-level are also globals.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Phillip J. Eby
At 10:27 AM 6/21/2006 -0700, Guido van Rossum wrote:
On 6/21/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
  At 09:55 AM 6/21/2006 -0700, Guido van Rossum wrote:
  BTW a switch in a class should be treated the same as a global switch.
  But what about a switch in a class in a function?
 
  Okay, now my head hurts.  :)

Welcome to the club. There's a Monty Python sketch appropriate...

Aha!  So *that's* why Jim Fulton is always going W.  :)


  A switch in a class doesn't need to be treated the same as a global switch,
  because locals()!=globals() in that case.

But that's not the discerning rule in my mind; the rule is, how to
define at function definition time.

Wa!  (i.e., my head hurts again :)


  I think the top-level is the only thing that really needs a special case
  vs. the general error if you use a local variable in the expression rule.

To the contrary, at the top level my preferred semantics don't care
because they don't use a hash.

The strict rules about locals apply when it occurs inside a function,
since then we eval the case expressions at function definition time,
when the locals are undefined. This would normally be good enough, but
I worry (a bit) about this case:

   y = 12
   def foo(x, y):
 switch x:
 case y: print something

which to the untrained observer (I care about untrained readers much
more than about untrained writers!) looks like it would print
something if x equals y, the argument, while in fact it prints
something if x equals 12.

I was thinking this should be rejected due to a local being in the 'case' 
expression.


  Actually, it might be simpler just to always reject local variables -- even
  at the top-level -- and be done with it.

Can't because locals at the top-level are also globals.

But you could also just use literals, and the behavior would then be 
consistent.  But I'm neither so enamored of that solution nor so against 
if/elif behavior that I care to argue further.

One minor point, though: what happens if we generate an if/elif for the 
switch, and there's a repeated case value?  The advantage of still using 
the hash-based code at the top level is that you still get an error for 
duplicating keys.

Ugh.  It still seems like the simplest implementation is to say that the 
lookup table is built at first use and that the case expressions may not 
refer to variables that are known to be bound in the current scope, or 
rebound in the case of the top level.  So the 'case y' example would be a 
compile-time error, as would my silly words example.  But code that only 
used constants at the top level would work.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Fredrik Lundh
Fredrik Lundh wrote:

 But in most cases the 'constant' is actually an expression involving a
 global, often even a global in another module. (E.g. sre_compile.py)
 The compiler will have a hard time proving that this is really a
 constant, so it won't optimize the code.
 
 unless we come up with a way to make it possible to mark an variable as 
 a constant.

such as the primary

 'constant' expr

which simply means that expr will be evaluated at function definition 
time, rather than at runtime.  example usage:

 var = expression
 if var == constant sre.FOO:
 ...
 elif var == constant sre.BAR:
 ...
 elif var in constant (sre.FIE, sre.FUM):
 ...

/F

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
 But that's not the discerning rule in my mind; the rule is, how to
 define at function definition time.

 Wa!  (i.e., my head hurts again :)

Um, wasn't this your proposal (to freeze the case expressions at
function definition time)?

   I think the top-level is the only thing that really needs a special case
   vs. the general error if you use a local variable in the expression 
   rule.
 
 To the contrary, at the top level my preferred semantics don't care
 because they don't use a hash.
 
 The strict rules about locals apply when it occurs inside a function,
 since then we eval the case expressions at function definition time,
 when the locals are undefined. This would normally be good enough, but
 I worry (a bit) about this case:
 
y = 12
def foo(x, y):
  switch x:
  case y: print something
 
 which to the untrained observer (I care about untrained readers much
 more than about untrained writers!) looks like it would print
 something if x equals y, the argument, while in fact it prints
 something if x equals 12.

 I was thinking this should be rejected due to a local being in the 'case'
 expression.

Me too. I guess I was just pointing out that just evaluating it in
the global scope would not give an error, just like this is valid (but
confusing):

y = 12
def foo(y=y):
  print y
y = 13
foo()  # prints 12

   Actually, it might be simpler just to always reject local variables -- 
   even
   at the top-level -- and be done with it.
 
 Can't because locals at the top-level are also globals.

 But you could also just use literals, and the behavior would then be
 consistent.  But I'm neither so enamored of that solution nor so against
 if/elif behavior that I care to argue further.

Yeah, but if you have names for your constants it would be a shame if
you couldn't use them because they happen to be defined in the same
scope.

 One minor point, though: what happens if we generate an if/elif for the
 switch, and there's a repeated case value?  The advantage of still using
 the hash-based code at the top level is that you still get an error for
 duplicating keys.

Sure. But the downside is that it's now actually *slower* than the
if/elif version, because it must evaluate all the case expressions.

 Ugh.  It still seems like the simplest implementation is to say that the
 lookup table is built at first use and that the case expressions may not
 refer to variables that are known to be bound in the current scope, or
 rebound in the case of the top level.  So the 'case y' example would be a
 compile-time error, as would my silly words example.  But code that only
 used constants at the top level would work.

I don't like first use because it seems to invite tricks.

The 'case y' example can be flagged as a compile time error with
enough compile-time analysis (we *know* all the locals after all).

IMO your silly words example should just pass (at the global level);
it's silly but not evil, and it's totally clear what it does if it
does anything at all (using the if/elif translation semantics; not
using the first-use semantics). That it doesn't refactor cleanly into
a function body isn't enough reason to forbid it.

I feel some kind of rule of thumb coming up regarding language design,
but I'm having a hard time saying it clearly. It's something about
making commonly written idioms easy to understand even for people
without a full understanding of the language, so that (a) people
generalizing from a few examples without too much help or prior
understanding won't go too far off, and (b) people who *do* care to
read and understand the language spec can always clearly find out wat
any particular thing means and know the pitfalls.

An example is assignment. Python lets you do things like

  x = 42
  y = x

and it all sounds completely reasonable. But Fortran/C/C++ programmers
beware, although the syntax is familiar, this is really a name-binding
statement, not a value-copying statement.

There are many other examples. Function and class definition for
example (look like definitions but are run-time constructs unlike in
most other languages). Etc.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Guido van Rossum
On 6/21/06, Fredrik Lundh [EMAIL PROTECTED] wrote:
 Fredrik Lundh wrote:

  But in most cases the 'constant' is actually an expression involving a
  global, often even a global in another module. (E.g. sre_compile.py)
  The compiler will have a hard time proving that this is really a
  constant, so it won't optimize the code.
 
  unless we come up with a way to make it possible to mark an variable as
  a constant.

 such as the primary

  'constant' expr

 which simply means that expr will be evaluated at function definition
 time, rather than at runtime.  example usage:

  var = expression
  if var == constant sre.FOO:
  ...
  elif var == constant sre.BAR:
  ...
  elif var in constant (sre.FIE, sre.FUM):
  ...

This gets pretty repetitive. One might suggest that 'case' could imply
'constant'...?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] TRUNK is UNFROZEN, but in FEATURE FREEZE

2006-06-21 Thread Josiah Carlson

Anthony Baxter [EMAIL PROTECTED] wrote:
 2.5b1 is out, so I'm declaring the SVN trunk unfrozen. Note, though, 
 that as we're now post-beta, we're in FEATURE FREEZE. 

Hey Raymond, any word on those binascii additions, or should I clean up
that struct patch and add in some tests?

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Nick Maclaren
Brett Cannon's and Neal Norwitz's replies appreciated and noted, but
responses sent by mail.


Nick Coghlan [EMAIL PROTECTED] wrote:

 Python 2.4's decimal module is, in essence, a floating point emulator based 
 on 
 the General Decimal Arithmetic specification.

Grrk.  Format and all?  Because, in software, encoding, decoding and
dealing with the special cases accounts for the vast majority of the
time.  Using a format and specification designed for implementation
in software is a LOT faster (often 5-20 times).

 If you want floating point mathematics that doesn't have insane platform 
 dependent behaviour, the decimal module is the recommended approach. By the 
 time Python 2.6 rolls around, we will hopefully have an optimized version 
 implemented in C (that's being worked on already).

Yes.  There is no point in building a wheel if someone else is doing it.
Please pass my name on to the people doing the optimisation, as I have
a lot of experience in this area and may be able to help.  But it is a
fairly straightforward (if tricky) task.

 That said, I'm not clear on exactly what changes you'd like to make to the 
 binary floating point type, so I don't know if I think they're a good idea or 
 not :)

Now, here it is worth posting a reponse :-)

The current behaviour follows C99 (sic) with some extra checking (e.g.
division by zero raises an exception).  However, this means that a LOT
of errors will give nonsense answers without comment, and there are a
lot of ways to 'lose' NaN values quietly - e.g. int(NaN).  That is NOT
good software engineering.  So:

Mode A:  follow IEEE 754R slavishly, if and when it ever gets into print.
There is no point in following C99, as it is too ill-defined, even if it
were felt desirable.  This should not be the default, because of the
flaws I mention above (see Kahan on Java).

Mode B:  all numerically ambiguous or invalid operations should raise
an exception - including pow(0,0), int(NaN) etc. etc.  There is a moot
point over whether overflow is such a case in an arithmetic that has
infinities, but let's skip over that one for now.

Mode C:  all numerically ambiguous or invalid operations should return
a NaN (or infinity, if appropriate).  Anything that would lose the error
indication would raise an exception.  The selection between modes B and
C could be done by a method on the class - with mode B being selected
if any argument had it set, and mode C otherwise.

Now, both modes B and C are traditional approaches to numerical safety,
and have the property that error indications can't be lost by accident,
though they make no guarantees that the answers make sense.  I am
agnostic about which is better, though mode B is a LOT better from the
debugging point of view, as you discover an error closer to where it
was made.

Heaven help us, there could be a mode D, which would be mode C but
with trace buffers.  They are another sadly neglected software
engineering technique, but let's not add every bell and whistle on
the shelf :-)


tjreedy [EMAIL PROTECTED] wrote:
 
  experience from times of yore is that emulated floating-point would
  be fast enough that few, if any, Python users would notice.
 
 Perhaps you should enquire on the Python numerical and scientific computing 
 lists to see how many feel differently.  I don't see how someone crunching 
 numbers hours per day could not notice a slowdown.

Oh, certainly, almost EVERYONE will feel differently!  But that is
not the point.  Those few of us remaining (and there are damn few) who
know how a fast emulated floating-point performs know that the common
belief that it is very slow is wrong.  I have both used and implemented
it :-)

The point is, as I mention above, you MUST use a software-friendly
format AND specification if you want performance.  IEEE 754 and IBM's
decimal pantechnichon are both extremely software-hostile.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  [EMAIL PROTECTED]
Tel.:  +44 1223 334761Fax:  +44 1223 334679
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Numerical robustness, IEEE etc.

2006-06-21 Thread Nick Maclaren
Michael Hudson [EMAIL PROTECTED] wrote:
 
  As I have posted to comp.lang.python, I am not happy with Python's
  numerical robustness - because it basically propagates the 'features'
  of IEEE 754 and (worse) C99. 
 
 That's not really now I would describe the situation today.

It is certainly the case in 2.4.2, however you would describe it.

  2) Because some people are dearly attached to the current behaviour,
  warts and all, and there is a genuine quandary of whether the 'right'
  behaviour is trap-and-diagnose, propagate-NaN or whatever-IEEE-754R-
  finally-specifies (let's ignore C99 and Java as beyond redemption),
 
 Why?  Maybe it's clear to you, but it's not totally clear to me, and
 it any case the discussion would be better informed for not being too
 dismissive.

Why which?  There are several things that you might be puzzled over.
And where can I start?  Part of the problem is that I have spent a LOT
of time in these areas in the past decades, and have been involved
with many of the relevant standards, and I don't know what to assume.

  there might well need to be options.  These can obviously be done by
  a command-line option, an environment variable or a float method.
  There are reasons to disfavour the last, but all are possible.  Which
  is the most Pythonesque approach?
 
 I have heard Tim say that there are people who would dearly like to be
 able to choose.  Environment variables and command line switches are
 not Pythonic.

All right, but what is?  Firstly, for something that needs to be
program-global?  Secondly, for things that don't need to be brings
up my point of adding methods to a built-in class.

 I'm interested in making Python's floating point story better, and
 have worked on a few things for Python 2.5 -- such as
 pickling/marshalling of special values -- but I'm not really a
 numerical programmer and don't like to guess what they need.

Ah.  I must get a snapshot, then.  That was one of the lesser things
on my list.  I have spent a lot of the past few decades in the numerical
programming arena, from many aspects.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  [EMAIL PROTECTED]
Tel.:  +44 1223 334761Fax:  +44 1223 334679
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Code coverage reporting.

2006-06-21 Thread Walter Dörwald
Titus Brown wrote:

 On Mon, Jun 19, 2006 at 08:37:30AM -0400, Benji York wrote:
 - Brett Cannon wrote:
 - But it does seem accurate; random checking of some modules that got high 
 - but not perfect covereage all seem to be instances where dependency 
 - injection would be required to get the tests to work since they were 
 - based on platform-specific things.
 - 
 - I don't know if we need it hooked into the buildbots (unless it is dirt 
 - cheap to generate the report).
 - 
 - It would be interesting to combine the coverage over several platforms 
 - and report that.
 
 Yes, I noticed that the platform specific stuff doesn't get covered, of
 course.  It's very easy to do, *if* there's any way to get the coverage
 database from a central location (or send it back to a central location).
 
 It might be interesting to run coverage analysis -- either figleaf or
 Ned Batchelder's module[0] -- once a week on select buildbot machines
 (one linux, one windows, one mac, or some such) and make the coverage
 databases available via something like a downloadable static file.  Then
 anyone could download those files and do Interesting Things with them.
 
 --titus
 
 [0] I'm sorry, I don't know how Walter Dorwald generates his coverage;
 if it's OSS, then it'd be better to use because it shows C code coverage
 as well.

The script at 
http://styx.livinglogic.de/~walter/python/coverage/PythonCodeCoverage.py 
definitely is open source, so feel free to use in any way you want. The 
web application front end though isn't open source. The SQL script to 
recreate the database can be found here: 
http://styx.livinglogic.de/~walter/python/coverage/coverage.sql

Servus,
Walter

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PyRange_New() alternative?

2006-06-21 Thread Ralf W. Grosse-Kunstleve
http://docs.python.org/dev/whatsnew/ports.html says:

  The PyRange_New() function was removed. It was never documented, never used
in the core code, and had dangerously lax error checking.

I use this function (don't remember how I found it; this was years ago),
therefore my code doesn't compile with 2.5b1 (it did OK before with 2.5a2). Is
there an alternative spelling for PyRange_New()?

Thank you in advance!

Ralf


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] doc for new restricted execution design for Python

2006-06-21 Thread Brett Cannon
I have been working on a design doc for restricted execution of Python
as part of my dissertation for getting Python into Firefox to replace
JavaScript on the web.  Since this is dealing with security and
messing that up can be costly, I am sending it to the list for any
possible feedback.

I have already run the ideas past Neal, Guido, Jeremy, and Alex and
everyone seemed to think the design was sound (thanks to them and Will
for attending my meeting on it and giving me feedback that helped to
shape this doc), so hopefully there are no major issues with the
design itself.  There are a couple of places (denoted with XXX) where
there is an open issue still.  Feedback on those would be great.

Anyway, here it is.  I am going to be offline most of tomorrow so I
probably won't get back to comments until Friday.

And just in case people are wondering, I plan on doing the
implementation in the open on a branch within Python's repository so
if this design works out it will end up in the core (as for when that
would land, I don't know, but hopefully for 2.6).

-


  Restricted Execution for Python
###

About This Document
=

This document is meant to lay out the general design for re-introducing a
restriced execution model for Python.  This document should provide one with
enough information to understand the goals for restricted execution, what
considerations were made for the design, and the actual design itself.  Design
decisions should be clear and explain not only why they were chosen but
possible drawbacks from taking that approach.


Goal
=

A good restricted execution model provides enough protection to prevent
malicious harm to come to the system, and no more.  Barriers should be
minimized so as to allow most code that does not do anything that would be
regarded as harmful to run unmodified.

An important point to take into consideration when reading this document is to
realize it is part of my (Brett Cannon's) Ph.D. dissertation.  This means it is
heavily geared toward the restricted execution when the interpreter is working
with Python code embedded in a web page.  While great strides have been taken
to keep the design general enough so as to allow all previous uses of the
'rexec' module [#rexec]_ to be able to use the new design, it is not the
focused goal.  This means if a design decision must be made for the embedded
use case compared to sandboxing Python code in a Python application, the former
will win out.

Throughout this document, the term resource is to represent anything that
deserves possible protection.  This includes things that have a physical
representation (e.g., memory) to things that are more abstract and specific to
the interpreter (e.g., sys.path).

When referring to the state of an interpreter, it is either trusted or
untrusted.  A trusted interpreter has no restrictions imposed upon any
resource.  An untrusted interpreter has at least one, possibly more, resource
with a restriction placed upon it.


.. contents::


Use Cases
/

All use cases are based on how many untrusted or trusted interpreters are
running in a single process.


When the Interpreter Is Embedded


Single Untrusted Interpreter


This use case is when an application embeds the interpreter and never has more
than one interpreter running.

The main security issue to watch out for is not having default abilities be
provided to the interpreter by accident.  There must also be protection from
leaking resources that the interpreter needs for general use underneath the
covers into the untrusted interpreter.


Multiple Untrusted Interpreters
---

When multiple interpreters, all untrusted at varying levels, need to be running
within a single application.  This is the key use case that this proposed
design is targetted for.

On top of the security issues from a single untrusted interpreter, there is one
additional worry.  Resources cannot end up being leaked into other interpreters
where they are given escalated rights.


Stand-Alone Python
==

When someone has written a Python program that wants to execute Python code in
an untrusted interpreter(s).  This is the use case that 'rexec' attempted to
fulfill.

The added security issues for this use case (on top of the ones for the other
use cases) is preventing something from the trusted interpreter leaking into an
untrusted interpreter and having elevated permissions.  With the multiple
untrusted interpreters one did not have to worry about preventing actions from
occurring that are disallowed for all untrusted interpreters.  With this use
case you do have to worry about the binary distinction between trusted and
untrusted interpreters running in the same 

[Python-Dev] ImportWarning flood

2006-06-21 Thread Ralf W. Grosse-Kunstleve
http://docs.python.org/dev/whatsnew/other-lang.html says:

 One error that Python programmers sometimes make is forgetting to 
 include an __init__.py module in a package directory. Debugging this
 mistake can be confusing, and usually requires running Python with the 
 -v switch to log all the paths searched. In Python 2.5, a new
 ImportWarning warning is raised when an import would have picked up a
 directory as a package but no __init__.py was found.

I am getting tons of ImportWarning: Not importing directory. See below for
examples. It is impractical for me to reorganize our directory structure. I'd
be busy for a week or more and people would probably scream at me because all
the paths have changed. Are there other options to get rid of the warnings?

Thanks!

Ralf



/net/rosie/scratch1/rwgk/dist/libtbx/libtbx/command_line/scons.py:1:
ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx':
missing __init__.py
  from libtbx.utils import Sorry
/net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43:
ImportWarning: Not importing directory
'/net/rosie/scratch1/rwgk/dist/cctbx/math': missing __init__.py
  from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
/net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43:
ImportWarning: Not importing directory
'/net/rosie/scratch1/rwgk/dist/scitbx/math': missing __init__.py
  from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
scons: Reading SConscript files ...
/net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/__init__.py:112:
ImportWarning: Not importing directory
'/net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/CVS': missing
__init__.py
  file, path, desc = imp.find_module(self.name, smpath)


/net/rosie/scratch1/rwgk/dist/phenix/phenix/__init__.py:1: ImportWarning: Not
importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py
  try: import libtbx.forward_compatibility
/net/rosie/scratch1/rwgk/dist/phenix/phenix/refinement/__init__.py:1:
ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/iotbx':
missing __init__.py
  import iotbx.phil
/net/rosie/scratch1/rwgk/dist/iotbx/iotbx/phil.py:1: ImportWarning: Not
importing directory '/net/rosie/scratch1/rwgk/dist/cctbx': missing __init__.py
  from cctbx import sgtbx
/net/rosie/scratch1/rwgk/dist/cctbx/cctbx/array_family/flex.py:1:
ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx':
missing __init__.py
  import scitbx.array_family.flex
/net/rosie/scratch1/rwgk/dist/scitbx/scitbx/array_family/flex.py:2:
ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/boost':
missing __init__.py
  import boost.optional
/net/rosie/scratch1/rwgk/dist/libtbx/libtbx/utils.py:226: ImportWarning: Not
importing directory '/net/rosie/scratch1/rwgk/dist/mmtbx': missing __init__.py
  try: module = __import__(module_path)

etc. etc.


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Switch statement

2006-06-21 Thread Talin
Phillip J. Eby wrote:
 At 09:55 AM 6/21/2006 -0700, Guido van Rossum wrote:
 
BTW a switch in a class should be treated the same as a global switch.
But what about a switch in a class in a function?
 
 
 Okay, now my head hurts.  :)
 
 A switch in a class doesn't need to be treated the same as a global switch, 
 because locals()!=globals() in that case.
 
 I think the top-level is the only thing that really needs a special case 
 vs. the general error if you use a local variable in the expression rule.
 
 Actually, it might be simpler just to always reject local variables -- even 
 at the top-level -- and be done with it.

I don't get what the problem is here. A switch constant should have 
exactly the bahavior of a default value of a function parameter. We 
don't seem to have too many problems defining functions at the module 
level, do we?

-- Talin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Allow assignments in 'global' statements?

2006-06-21 Thread Talin
I'm sure I am not the first person to say this, but how about:

global x = 12

(In other words, declare a global and assign a value to it - or another 
way of saying it is that the 'global' keyword acts as an assignment 
modifier.)

-- Talin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ImportWarning flood

2006-06-21 Thread Guido van Rossum
On 6/21/06, Ralf W. Grosse-Kunstleve [EMAIL PROTECTED] wrote:
 I am getting tons of ImportWarning: Not importing directory. See below for
 examples. It is impractical for me to reorganize our directory structure. I'd
 be busy for a week or more and people would probably scream at me because all
 the paths have changed. Are there other options to get rid of the warnings?

Check out the -W command line option and the warnings module. These
document how to suppress warnings.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] ImportWarning flood

2006-06-21 Thread Ralf W. Grosse-Kunstleve
--- Guido van Rossum [EMAIL PROTECTED] wrote:

 On 6/21/06, Ralf W. Grosse-Kunstleve [EMAIL PROTECTED] wrote:
  I am getting tons of ImportWarning: Not importing directory. See below
 for
  examples. It is impractical for me to reorganize our directory structure.
 I'd
  be busy for a week or more and people would probably scream at me because
 all
  the paths have changed. Are there other options to get rid of the warnings?
 
 Check out the -W command line option and the warnings module. These
 document how to suppress warnings.

Thanks!

This does the trick:

python -W'ignore:Not importing directory'

But this doesn't:

python -W'ignore:Not importing directory:ImportWarning'

I tried a bunch of variations without success. A few examples here would be
very valuable:

http://docs.python.org/lib/warning-filter.html

Also, the magic incantation to silence the warnings would be very helpful here:

http://docs.python.org/dev/whatsnew/other-lang.html

Is there a way to set the warning options via an environment variable?
Otherwise I am forced to use a wrapper or aliases.


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Weekly Python Patch/Bug Summary

2006-06-21 Thread Kurt B. Kaiser
Patch / Bug Summary
___

Patches :  378 open ( +3) /  3298 closed (+34) /  3676 total (+37)
Bugs:  886 open (-24) /  5926 closed (+75) /  6812 total (+51)
RFE :  224 open ( +7) /   227 closed ( +7) /   451 total (+14)

New / Reopened Patches
__

Improve super() objects support for implicit method calls  (2006-05-31)
   http://python.org/sf/1498363  opened by  Collin Winter

Improve itertools.starmap  (2006-05-31)
   http://python.org/sf/1498370  opened by  Collin Winter

Change *args from a tuple to list  (2006-05-31)
   http://python.org/sf/1498441  opened by  Collin Winter

Correctly unpickle exceptions  (2006-06-01)
CLOSED http://python.org/sf/1498571  opened by  Žiga Seilnacht

Fault in XMLRPC not always returned to client  (2006-05-31)
CLOSED http://python.org/sf/1498627  opened by  Daniel Savard

Generate from Unicode database instead of manualy coding.  (2006-06-01)
   http://python.org/sf/1498930  opened by  Anders Chrigström

Optimise in operations on tuples of consts  (2006-06-01)
CLOSED http://python.org/sf/1499095  opened by  Collin Winter

Fix for memory leak in WindowsError_str  (2006-06-02)
CLOSED http://python.org/sf/1499797  opened by  Žiga Seilnacht

Alternate RFC 3896 compliant URI parsing module  (2006-06-05)
   http://python.org/sf/1500504  opened by  Nick Coghlan

Remove dependencies on the sets module  (2006-06-04)
   http://python.org/sf/1500609  opened by  Collin Winter

(py3k) Remove the sets module  (2006-06-04)
   http://python.org/sf/1500611  opened by  Collin Winter

Remove the repr()-backticks  (2006-06-04)
   http://python.org/sf/1500623  opened by  Collin Winter

wm_attributes doesn't take keyword arguments  (2006-06-04)
   http://python.org/sf/1500773  opened by  Greg Couch

AF_NETLINK support for socket module  (2006-06-05)
CLOSED http://python.org/sf/1500815  opened by  lplatypus

Cyclic garbage collection support for slices  (2006-06-05)
CLOSED http://python.org/sf/1501180  opened by  Žiga Seilnacht

Fix Bug #1339007 - shelve.open('non-existant-file', 'r')  (2006-06-06)
CLOSED http://python.org/sf/1501534  opened by  Jeung Mun Sic

syntax errors on continuation lines  (2006-06-06)
   http://python.org/sf/1501979  opened by  Roger Miller

Remove randomness from test_exceptions  (2006-06-07)
CLOSED http://python.org/sf/1501987  opened by  Žiga Seilnacht

Conditional compilation of zlib.(de)compressobj.copy  (2006-06-08)
CLOSED http://python.org/sf/1503046  opened by  Chris AtLee

Allow Empty Subscript List Without Parentheses  (2006-06-09)
CLOSED http://python.org/sf/1503556  opened by  Noam Raphael

Tiny patch to stop make spam  (2006-06-09)
   http://python.org/sf/1503717  opened by  Chris AtLee

Rough documentation for xml.etree.ElementTree  (2006-06-10)
   http://python.org/sf/1504046  opened by  Fredrik Lundh

Patch for 1496501 tarfile opener order  (2006-06-10)
   http://python.org/sf/1504073  opened by  Jack Diederich

Switch syntax (partial PEP 275)  (2006-06-11)
   http://python.org/sf/1504199  opened by  Thomas Lee

winerror module  (2006-06-13)
   http://python.org/sf/1505257  opened by  M.-A. Lemburg

curses.resizeterm()  (2006-06-15)
CLOSED http://python.org/sf/1506645  opened by  Walter Dörwald

Patch for 1506758 - popen2/subprocess MAXFD MemoryErrors  (2006-06-15)
   http://python.org/sf/1506760  opened by  Peter Vetere

Use a set to keep interned strings  (2006-06-15)
   http://python.org/sf/1507011  opened by  Alexander Belopolsky

tarfile extraction does not honor umask  (2006-06-16)
   http://python.org/sf/1507247  opened by  Faik Uygur

improve object.c and abstract.c exception messages  (2006-06-17)
CLOSED http://python.org/sf/1507676  opened by  Georg Brandl

transparent gzip compression in liburl2  (2006-06-19)
   http://python.org/sf/1508475  opened by  Jakob Truelsen

uuid documentation  (2006-06-20)
   http://python.org/sf/1508766  opened by  George Yoshida

Make Lib/test/regrtest.py NetBSD 3 aware.  (2006-06-20)
CLOSED http://python.org/sf/1509001  opened by  Matt Fleming

MS Toolkit Compiler no longer available  (2006-06-20)
   http://python.org/sf/1509163  opened by  Paul Moore

skip tests in test_socket__ssl when connection refused  (2006-06-20)
CLOSED http://python.org/sf/1509404  reopened by  bcannon

skip tests in test_socket__ssl when connection refused  (2006-06-20)
CLOSED http://python.org/sf/1509404  opened by  Brett Cannon

Small fix for sqlite3 test suite  (2006-06-20)
CLOSED http://python.org/sf/1509584  opened by  Gerhard Häring

tarfile stops iteration with some longfiles  (2006-06-21)
CLOSED http://python.org/sf/1509889  opened by  Faik Uygur

Patches Closed
__

Possible fix to #1334662 (int() wrong answers)  (2006-03-31)
   http://python.org/sf/1462361  closed by  gbrandl

Correctly unpickle exceptions  (2006-05-31)
   http://python.org/sf/1498571  closed by  gbrandl

Fault in