Re: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions...

2009-02-22 Thread Carl Johnson
Nick Coghlan wrote:

 Josiah Carlson wrote:
  Similarly, a 3.x list comprehension [i*i for i in x] is very roughly
  translated as:
 
   def _lc(arg):
 result = []
 for i in arg:
   result.append(i*i)
 return result
 
   expr_value = _lc(x)
 
  I was under the impression that in 3.x, it was equivalent to
  list(genexp) .  Which would explain the difference between 2.6 and
  3.0.

 Semantically, the two translations end up being the same.

Actually, they're not quite the same. list(genexp) will swallow any
StopIteration exceptions that end up getting thrown inside the
generator, but even in Python 3, list comprehensions will not catch
StopIteration. So for explanatory purposes, thinking of a list
comprehension as a function that returns a list is more useful than
thinking of it as list(genexp).

-- Carl
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Attention Bazaar mirror users

2009-02-22 Thread Michael Foord

Steve Holden wrote:

Steven Bethard wrote:
  

On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore p.f.mo...@gmail.com wrote:


PS Just for my own information, am I correct in thinking that it is
*only* Bazaar in the (D)VCS world that has this problem, to any real
extent? I know old Mercurial clients can interact with newer servers
(ie, the wire protocol hasn't changed), I'm fairly sure that older
Subversion clients can talk to newer servers (at least, I've never
cared what client version I'm running).
  

I've definitely had an SVN server tell me that I needed to upgrade my
client to 1.5.



Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
client to 1.5, which is a little tedious = particularly as the
information comes in the form of a hard-to-decipher error message and a
refusal to work. So much for backward compatibility.
  


Hmm... I've been using 1.5 clients with 1.4 and earlier servers for 
quite a while now. My guess is that you deciphered the error message wrong!


A working copy created by a 1.5 client can only be manipulated by a 1.5 
client (I sometimes have several clients on windows boxen) but work fine 
with earlier servers in my experience.


Michael


Fortunately I still had the SVN 1.4 client on the Ubuntu machine that
was hosting the Samba shares, so I could use ssh and the command line to
maintain things.

regards
 Steve
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Attention Bazaar mirror users

2009-02-22 Thread Steve Holden
Michael Foord wrote:
 Steve Holden wrote:
 Steven Bethard wrote:
  
 On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore p.f.mo...@gmail.com wrote:

 PS Just for my own information, am I correct in thinking that it is
 *only* Bazaar in the (D)VCS world that has this problem, to any real
 extent? I know old Mercurial clients can interact with newer servers
 (ie, the wire protocol hasn't changed), I'm fairly sure that older
 Subversion clients can talk to newer servers (at least, I've never
 cared what client version I'm running).
   
 I've definitely had an SVN server tell me that I needed to upgrade my
 client to 1.5.
 

 Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
 client to 1.5, which is a little tedious = particularly as the
 information comes in the form of a hard-to-decipher error message and a
 refusal to work. So much for backward compatibility.
   
 
 Hmm... I've been using 1.5 clients with 1.4 and earlier servers for
 quite a while now. My guess is that you deciphered the error message wrong!
 
 A working copy created by a 1.5 client can only be manipulated by a 1.5
 client (I sometimes have several clients on windows boxen) but work fine
 with earlier servers in my experience.
 
You are correct. In fact I got over the issue by using the 1.4 client
from Cygwin.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Reviving restricted mode?

2009-02-22 Thread Guido van Rossum
I've received some enthusiastic emails from someone who wants to
revive restricted mode. He started out with a bunch of patches to the
CPython runtime using ctypes, which he attached to an App Engine bug:

http://code.google.com/p/googleappengine/issues/detail?id=671

Based on his code (the file secure.py is all you need, included in
secure.tar.gz) it seems he believes the only security leaks are
__subclasses__, gi_frame and gi_code. (I have since convinced him that
if we add restricted guards to these attributes, he doesn't need the
functions added to sys.)

I don't recall the exploits that Samuele once posted that caused the
death of rexec.py -- does anyone recall, or have a pointer to the
threads?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Benjamin Peterson
On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum gu...@python.org wrote:
 I've received some enthusiastic emails from someone who wants to
 revive restricted mode. He started out with a bunch of patches to the
 CPython runtime using ctypes, which he attached to an App Engine bug:

 http://code.google.com/p/googleappengine/issues/detail?id=671

 Based on his code (the file secure.py is all you need, included in
 secure.tar.gz) it seems he believes the only security leaks are
 __subclasses__, gi_frame and gi_code. (I have since convinced him that
 if we add restricted guards to these attributes, he doesn't need the
 functions added to sys.)

We have to remember that it's quite trivial to segfault the
interpreter with pure Python. (See Lib/test/crashers)

Even if this patch manages to plug all the holes in the current
Python, do we really want to commit our selves to maintaining it
through language evolution which will surely introduce new subtle ways
to circumvent the guard?

IMO, the only fairly close to fool proof method of running restricted
python is through something like the PyPy sandbox where all os level
calls have to be checked by the master process.


 I don't recall the exploits that Samuele once posted that caused the
 death of rexec.py -- does anyone recall, or have a pointer to the
 threads?

It was broken by the introduction of new-style classes:

http://mail.python.org/pipermail/python-dev/2002-December/031160.html



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread James Pye

On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote:
The other approach is having pickle contain code known not to be  
overridden by anyone, import _pypickle for stuff that may be  
overridden, and then import _pickle for whatever is available. This  
approach has the perk of using a standard practice for how to pull  
in different implementation. But the drawback, thanks to how globals  
are bound, is that any code pulled in from _pickle/_pypickle will  
not be able to call into other optimized code; it's a take or leave  
it once the call chain enters one of those modules as they will  
always call the implementations in the module they originate from.



fwiw,

I believe this is the approach that I've been using when I'm faced  
with the need to optimize code, but still want to retain a pure-Python  
version. Thankfully, I haven't had a need for implementation  
intersections(well, it almost works. I think. ;) for access to common  
module globals as the optimizations tend to be simple transformations  
or isolated classes. However, if I were faced with the problem of  
needing some common global data/functionality, I'd probably put it in  
yet-another-module and just import it explicitly in each  
implementation. Sure, it seems like it might be annoying, but so is  
maintaining multiple implementations. ;)


Specifically:

pbuffer.py - The python implementation
buffer.c - cbuffer.so - The c implementation
buffer.py - The abstraction module, trying to import the contents of  
the fast one first.


And in my unittest:

if c_buffer_module is not None:
  class c_buffer(buffer_test, unittest.TestCase):
bufferclass = c_buffer_module.pq_message_stream

class p_buffer(buffer_test, unittest.TestCase):
  bufferclass = p_buffer_module.pq_message_stream

Of course, buffer_test is not invoked because it's not a TestCase.


However, Aahz is probably right about this thread belonging elsewhere.?
Hrm, day old, maybe it's been moved already.. sigh. :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Samuele Pedroni

Guido van Rossum wrote:

I've received some enthusiastic emails from someone who wants to
revive restricted mode. He started out with a bunch of patches to the
CPython runtime using ctypes, which he attached to an App Engine bug:

http://code.google.com/p/googleappengine/issues/detail?id=671

Based on his code (the file secure.py is all you need, included in
secure.tar.gz) it seems he believes the only security leaks are
__subclasses__, gi_frame and gi_code. (I have since convinced him that
if we add restricted guards to these attributes, he doesn't need the
functions added to sys.)

I don't recall the exploits that Samuele once posted that caused the
death of rexec.py -- does anyone recall, or have a pointer to the
threads?

  
I don't have much time these days, for sure not until pycon us, to look 
at the proposed code.


A remark though, in many cases people want to use restricted execution 
to allow trusted and untrusted code to live side by side. In such a 
scenario the issue is not only how to prevent for example the untrusted 
code to get hold of open() but also how to deal with the security issues 
at the boundary when untrusted code calls into trusted code. AFAIK E 
provides and incorporate a lot of thinking around the concept of guards, 
a sort of generalized runtime type checking with very strong 
guarantees, in a language like Java instead type checking and final 
classes can be used to provide the required safety. Python instead over 
the years has grown more hooks for object to cheat about their types. In 
general safely programming such trusted code in python will be delicate 
and slightly cumbersome. Think for example of thinhs that can be done by 
objects redefining equality wrt data structures that old sensitive 
information unless such arguments are blocked.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Steven Bethard
On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote:
 But there is another issue with this: the pure Python code will never call
 the extension code because the globals will be bound to _pypickle and not
 _pickle. So if you have something like::

   # _pypickle
   def A(): return _B()
   def _B(): return -13

   # _pickle
   def _B(): return 42

   # pickle
   from _pypickle import *
   try: from _pickle import *
   except ImportError: pass

 If you import pickle and call pickle.A() you will get -13 which is not what
 you are after.

Maybe I've missed this and someone else already suggested it, but
couldn't we provide a (probably C-coded) function
``replace_globals(module, globals)`` that would, say, replace the
globals in _pypickle with the globals in pickle? Then you could write
something like::

from _pypickle import *
try:
from _pickle import *
module = __import__('_pickle')
except ImportError:
module = __import__('_pypickle')
replace_globals(module, globals())

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Michael Foord

Steven Bethard wrote:

On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote:
  

But there is another issue with this: the pure Python code will never call
the extension code because the globals will be bound to _pypickle and not
_pickle. So if you have something like::

  # _pypickle
  def A(): return _B()
  def _B(): return -13

  # _pickle
  def _B(): return 42

  # pickle
  from _pypickle import *
  try: from _pickle import *
  except ImportError: pass

If you import pickle and call pickle.A() you will get -13 which is not what
you are after.



Maybe I've missed this and someone else already suggested it, but
couldn't we provide a (probably C-coded) function
``replace_globals(module, globals)`` that would, say, replace the
globals in _pypickle with the globals in pickle? Then you could write
something like::

from _pypickle import *
try:
from _pickle import *
module = __import__('_pickle')
except ImportError:
module = __import__('_pypickle')
replace_globals(module, globals())

Steve
  


Swapping out module globals seems to be a backwards way to do things to 
me. Why not have one set of tests that test the low level APIs 
(identical tests whether they are written in C or Python) - and as they 
live in their own module are easy to test in isolation. And then a 
separate set of tests for the higher level APIs, which can even mock out 
the low level APIs they use. There shouldn't be any need for switching 
out objects in the scope of the lower level APIs - that seems like a 
design smell to me.


Michael

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Antoine Pitrou
Hello,

The Python version of IO lib has taken the stance that all errors happening in
the destructor of an IO object are silenced. Here is the relevant code in 
IOBase:

def __del__(self) - None:
Destructor.  Calls close().
# The try/except block is in case this is called at program
# exit time, when it's possible that globals have already been
# deleted, and then the close() call might fail.  Since
# there's nothing we can do about such failures and they annoy
# the end users, we suppress the traceback.
try:
self.close()
except:
pass

However, this seems very unreliable and dangerous to me. Users will not be
warned if their IO objects are not closed properly (which might be due to
programming errors), and data possibly gets corrupted.

I would like to change this behaviour so that the try/except enclosure above
is removed, letting __del__ at least print those (unraisable) errors on the
console. Of course I also advocate doing so in the C version of the IO lib.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 9:54 AM, Benjamin Peterson benja...@python.org wrote:
 On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum gu...@python.org wrote:
 I've received some enthusiastic emails from someone who wants to
 revive restricted mode. He started out with a bunch of patches to the
 CPython runtime using ctypes, which he attached to an App Engine bug:

 http://code.google.com/p/googleappengine/issues/detail?id=671

 Based on his code (the file secure.py is all you need, included in
 secure.tar.gz) it seems he believes the only security leaks are
 __subclasses__, gi_frame and gi_code. (I have since convinced him that
 if we add restricted guards to these attributes, he doesn't need the
 functions added to sys.)

 We have to remember that it's quite trivial to segfault the
 interpreter with pure Python. (See Lib/test/crashers)

I know it well, but for this particular use case that doesn't matter.
Tav is interested in using this on app engine, which doesn't care
about segfaults -- the process is simply restarted, nobody gains
access to information they shouldn't have. App engine does care about
overwriting memory, but none of the crashers get that far in the
configuration on app engine.

 Even if this patch manages to plug all the holes in the current
 Python, do we really want to commit our selves to maintaining it
 through language evolution which will surely introduce new subtle ways
 to circumvent the guard?

You'd have to talk to Tav about this (I've CC'ed him on this message).
He seems quite convinced that his current approach is secure, all he
wants is a way to do the equivalent in app engine (where ctypes is not
supported and never will be).

 IMO, the only fairly close to fool proof method of running restricted
 python is through something like the PyPy sandbox where all os level
 calls have to be checked by the master process.

Trust me, app engine has something similar -- Tav's desire is just to
provide an added barrier between his app and code that his app will
let users run within a context it provides. He's relying (forced to
rely :-) on app engine's sandbox for protection against illegal
syscalls.

 I don't recall the exploits that Samuele once posted that caused the
 death of rexec.py -- does anyone recall, or have a pointer to the
 threads?

 It was broken by the introduction of new-style classes:

 http://mail.python.org/pipermail/python-dev/2002-December/031160.html

I think there were other threads where Samule showed some quite
devious ways to access the os module. I don't see any posts by Samuele
in that thread (and he just replied that he's too busy). The attack
demonstrated at the top of that thread isn't particularly interesting
-- it just shows that class-based security doesn't work so well, but
that doesn't mean some other approach based more on functions couldn't
work.

For Tav's benefit, I think it would be good to at least add
IsRestricted checks to __subclasses__(), go_code and gi_frame --
that's a trivial patch and if he believes it's enough he can create a
sandbox on app engine and invite people to try to break out of it...
If someone succeeds, the damage is limited by app engine's own
perimeter defenses (which haven't been broken through since it was
released 9 months ago).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 11:02 AM, Antoine Pitrou solip...@pitrou.net wrote:
 The Python version of IO lib has taken the stance that all errors happening in
 the destructor of an IO object are silenced. Here is the relevant code in 
 IOBase:

def __del__(self) - None:
Destructor.  Calls close().
# The try/except block is in case this is called at program
# exit time, when it's possible that globals have already been
# deleted, and then the close() call might fail.  Since
# there's nothing we can do about such failures and they annoy
# the end users, we suppress the traceback.
try:
self.close()
except:
pass

 However, this seems very unreliable and dangerous to me. Users will not be
 warned if their IO objects are not closed properly (which might be due to
 programming errors), and data possibly gets corrupted.

OTOH the is a much larger category of false positives, where a close()
call raise an exception for some spurious reason (see the quoted
comment) but no harm is done; in the past the tracebacks printed for
__del__ by default have caused nothing but confuse users (who think
something is seriously wrong but have no way to debug it).

The svn history of those lines may have more pointers.

 I would like to change this behaviour so that the try/except enclosure above
 is removed, letting __del__ at least print those (unraisable) errors on the
 console. Of course I also advocate doing so in the C version of the IO lib.

I'd be very wary of this change. That code has enough comments to make
me believe that I put it in for a very good reason...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Greg Ewing

Antoine Pitrou wrote:


I would like to change this behaviour so that the try/except enclosure above
is removed, letting __del__ at least print those (unraisable) errors on the
console. Of course I also advocate doing so in the C version of the IO lib.


I would hope that the C version could at least ensure that
buffers are flushed properly without having to rely on any
global variables, so that this would not be so much of an
issue.

When we were using stdio, we could rely on the C runtime
to flush buffers unless something very drastic has gone
wrong. Bypassing stdio means we are taking that
responsibility on ourselves.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Greg Ewing

Guido van Rossum wrote:


Tav is interested in using this on app engine, which doesn't care
about segfaults -- the process is simply restarted, nobody gains
access to information they shouldn't have. App engine does care about
overwriting memory,


That doesn't make sense -- how can something not care
about segfaults, but care about memory overwriting?
If something is capable of causing a segfault, you
can't be sure it won't just corrupt memory in some
way that doesn't segfault but causes some other
problem.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Antoine Pitrou
Greg Ewing greg.ewing at canterbury.ac.nz writes:
 
 I would hope that the C version could at least ensure that
 buffers are flushed properly without having to rely on any
 global variables, so that this would not be so much of an
 issue.

The C version cleans up after itself just fine :)
The issue is when someone defines a Python class derived from IOBase or one of
its descendants (RawIOBase, BufferedIOBase, TextIOBase), and overrides the
close() method.
(see e.g. SocketIO in socket.py, which incidentally defined its own __del__
while it could perhaps rely on RawIOBase to do the right thing)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 12:10 PM, Greg Ewing
greg.ew...@canterbury.ac.nz wrote:
 Tav is interested in using this on app engine, which doesn't care
 about segfaults -- the process is simply restarted, nobody gains
 access to information they shouldn't have. App engine does care about
 overwriting memory,

 That doesn't make sense -- how can something not care
 about segfaults, but care about memory overwriting?
 If something is capable of causing a segfault, you
 can't be sure it won't just corrupt memory in some
 way that doesn't segfault but causes some other
 problem.

To be more precise, we don't care about crashes caused by NULL pointer
dereferencing. Most of the demonstrated crashers work by causing a
NULL pointer dereference. Since that crashes immediately, there is no
possibility for a further exploit.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread tav
Hey guys,

  benjamin Even if this patch manages to plug all the holes in the
  benjamin current Python, do we really want to commit our
  benjamin selves to maintaining it through language evolution
  benjamin which will surely introduce new subtle ways to
  benjamin circumvent the guard?

If it would be helpful, I am happy to maintain this as Python evolves.

I've already been maintaining the PJE-inspired ctypes-based approach
and monkeypatches for various Python versions for a while now. See
secure.py, secure25.py, secure26.py and secure30.py in:

  
http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util

Also, my plans for world domination depend on a secure Python, so I
have the necessary incentives ;p

  sameule I don't have much time these days, for sure not
  samuele until pycon us, to look at the proposed code.

Thanks in advance if/when you get the time for this Samuele!

  samuele E provides and incorporate a lot of thinking
  samuele around [snip]

The functions based approach I am taking is very much taken from E and
inspired by an insight that Ka-Ping Yee had on Python-Dev years ago.

See http://www.erights.org/elib/capability/ode/index.html for a direct
parallel to the approach I've taken...

  guido For Tav's benefit, I think it would be good to at
  guido least add IsRestricted checks to
  guido __subclasses__(), gi_code and gi_frame --
  guido that's a trivial patch and if he believes it's
  guido enough he can create a sandbox on app engine
  guido and invite people to try to break out of it... If
  guido someone succeeds

If someone succeeds...

...My missus might end up leaving me on account of so much crying ;p

Seriously though, it's a relatively risk-free approach. The only
person who stands to lose out is me if I'm wrong =)

In the worst case scenario, this approach would help identify other
leak attributes/methods -- which I'm hoping won't be found.

And, in an ideal scenario, we'd have the basis for secure Python
interpreter/programming... which, together with PyPy's sandboxed
interpreter, would seriously rock!

-- 
enthusiastically, tav

plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 11:32 AM, Antoine Pitrou solip...@pitrou.net wrote:
 Guido van Rossum guido at python.org writes:
 OTOH the is a much larger category of false positives, where a close()
 call raise an exception for some spurious reason (see the quoted
 comment) but no harm is done; in the past the tracebacks printed for
 __del__ by default have caused nothing but confuse users (who think
 something is seriously wrong but have no way to debug it).

 The svn history of those lines may have more pointers.

 Well this code dates back to the first checkin in the py3k branch. Apparently
 the old p3yk branch is not there anymore...

OK, then we'll just have to understand the big comment in the code.

 I understand the missing globals on shutdown problem, but the error may also 
 be
 a legitimate programming error, where a close() implementation fails for
 whatever reason. At least displaying the error may encourage the programmer to
 strengthen his implementation.

 How about the following compromise:

try:
closed = self.closed
except:
# Object is in an unusable state, don't attempt anything
pass
else:
if not closed:
self.close()

No. Trust me. It is not always possible to strengthen the
implementation. (At least not until we get rid of the replace all
globals with None upon module deletion rule.) Your legitimate
programming error is purely hypothetical, while the unactionable,
confusing traceback caused by missing globals is real. If people want
an exception from close() they can call it explicitly or use a context
manager. When a file is closed implicitly by __del__, there is a
chance of an impossible-to-avoid spurious traceback. I have dealt with
such tracebacks in real life and there is nothing that can be done
about them except ignore them. If you don't ignore them, users living
1000s of miles away and years later than the original programmer will
be upset by spurious tracebacks that worry them to death.

If you want to ensure buffers are flushed, why not call self.flush()
for writable files outside the try/except? If flush() fails it *is* a
real problem.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Tracker archeology

2009-02-22 Thread Rodrigo Bernardo Pimentel
[Sorry for stepping in so late]

On Thu, Feb 12 2009 at 02:05:23PM BRST, Daniel (ajax) Diniz 
aja...@gmail.com wrote:
 Victor Stinner wrote:
  Oh, I realized that there is a component called Unicode. So it should be
  possible to write a request to list all issues related to unicode.
 
 Nice, I'll add set this component for issues that don't have it. I can
 still add people to these issues, if they want.

I think this one might be considered a Unicode issue:

http://bugs.python.org/issue1293741 doctest runner cannot handle non-ascii
characters


rbp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Brett Cannon
On Sat, Feb 21, 2009 at 20:12, Aahz a...@pythoncraft.com wrote:

 On Sat, Feb 21, 2009, Brett Cannon wrote:
  On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote:
  On Sat, Feb 21, 2009, Brett Cannon wrote:
 
  I am seeing two approaches emerging. One is where pickle contains all
  Python code and then uses something like use_extension to make sure
  the original Python objects are still reachable at some point. This
  has the drawback that you have to use some function to make the
  extensions happen and there is some extra object storage.
 
  The other approach is having pickle contain code known not to
  be overridden by anyone, import _pypickle for stuff that may be
  overridden, and then import _pickle for whatever is available. This
  approach has the perk of using a standard practice for how to pull in
  different implementation. But the drawback, thanks to how globals are
  bound, is that any code pulled in from _pickle/_pypickle will not be
  able to call into other optimized code; it's a take or leave it once
  the call chain enters one of those modules as they will always call
  the implementations in the module they originate from.
 
  To what extent do we care about being able to select Python-only on a
  per-module basis, particularly in the face of threaded imports?  That
 is,
  we could have a sys.python_only attribute that gets checked on import.
  That's simple and direct, and even allows per-module switching if the
  application really cares and import doesn't need to worry about threads.
 
  Alternatively, sys.python_only could be a set, but that gets ugly about
  setting from the application.  (The module checks to see whether it's
  listed in sys.python_only.)
 
  Maybe we should move this discussion to python-ideas for now to kick
  around really oddball suggestions?
 
  This is all about testing. If a change is made to some extension code it
  should be mirrored in the Python code and vice-versa.

 Okay, I don't see how that is a response to my suggestion -- I can
 imagine that someone might want to test a combination of pure-Python and
 binary libraries.


I don't want to move it because this isn't some idea for a new feature that
may or may not be useful; this isn't an idea, it's needed.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Martin v. Löwis
 I've already been maintaining the PJE-inspired ctypes-based approach
 and monkeypatches for various Python versions for a while now. See
 secure.py, secure25.py, secure26.py and secure30.py in:
 
   
 http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util

What is the objective of this code? Is it a complete sandbox?
If not, is a complete sandbox based on it available somehow for
review?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Brett Cannon
On Sun, Feb 22, 2009 at 10:29, Michael Foord fuzzy...@voidspace.org.ukwrote:

 Steven Bethard wrote:

 On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote:


 But there is another issue with this: the pure Python code will never
 call
 the extension code because the globals will be bound to _pypickle and not
 _pickle. So if you have something like::

  # _pypickle
  def A(): return _B()
  def _B(): return -13

  # _pickle
  def _B(): return 42

  # pickle
  from _pypickle import *
  try: from _pickle import *
  except ImportError: pass

 If you import pickle and call pickle.A() you will get -13 which is not
 what
 you are after.



 Maybe I've missed this and someone else already suggested it, but
 couldn't we provide a (probably C-coded) function
 ``replace_globals(module, globals)`` that would, say, replace the
 globals in _pypickle with the globals in pickle? Then you could write
 something like::

from _pypickle import *
try:
from _pickle import *
module = __import__('_pickle')
except ImportError:
module = __import__('_pypickle')
replace_globals(module, globals())

 Steve



 Swapping out module globals seems to be a backwards way to do things to me.


I agree; I would rather muck with sys.modules at that point.

Why not have one set of tests that test the low level APIs (identical tests
 whether they are written in C or Python) - and as they live in their own
 module are easy to test in isolation. And then a separate set of tests for
 the higher level APIs, which can even mock out the low level APIs they use.
 There shouldn't be any need for switching out objects in the scope of the
 lower level APIs - that seems like a design smell to me.


 That's possible. As I have said, my only worry with the separate
py/extension module approach is that any time someone wants to do an
extension version of something the Python code will need to be moved.

But at this point I am honestly burning out on this topic (got a lot on my
plate right now) so I am going to let somebody else lead this to the finish
line.  =)

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Antoine Pitrou
Guido van Rossum guido at python.org writes:
 
 If you want to ensure buffers are flushed, why not call self.flush()
 for writable files outside the try/except? If flush() fails it *is* a
 real problem.

Well, my concern is less about flushing writable files (they will be
implemented in C and the flushing will probably be ok) than Python-written 
file-like types, like SocketIO or any third-party class.

Anyway, let's keep silencing exceptions and we'll see if anyone complains
about it.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Silencing IO errors on del/dealloc?

2009-02-22 Thread Daniel (ajax) Diniz
Antoine Pitrou wrote:
 Guido van Rossum guido at python.org writes:
 The svn history of those lines may have more pointers.

 Well this code dates back to the first checkin in the py3k branch. Apparently
 the old p3yk branch is not there anymore...

The history is available (see below), but tells nothing that would be
useful/relevant.

Daniel


History of io.py is available on ViewVC:
http://svn.python.org/view/python/branches/p3yk/Lib/io.py?view=logpathrev=56853

It's possible to checkout that as a peg revision:
svn  co -r54910 http://svn.python.org/projects/python/branches/p3yk/Lib/@r54910

Then, svn blame tells where it comes from:
 54728 guido.van.rossum def __del__(self) - None:
 54728 guido.van.rossum Destructor.  Calls close().
 54728 guido.van.rossum # The try/except block is in case this
is called at program
 54728 guido.van.rossum # exit time, when it's possible that
globals have already been
 54728 guido.van.rossum # deleted, and then the close() call
might fail.  Since
 54728 guido.van.rossum # there's nothing we can do about such
failures and they annoy
 54728 guido.van.rossum # the end users, we suppress the traceback.
 54728 guido.van.rossum try:
 54728 guido.van.rossum self.close()
 54728 guido.van.rossum except:
 54728 guido.van.rossum pass

And here's the log for that rev:
http://svn.python.org/view?view=revrevision=54728
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Ivan Krstić

On Feb 22, 2009, at 5:15 PM, Martin v. Löwis wrote:

What is the objective of this code? Is it a complete sandbox?
If not, is a complete sandbox based on it available somehow for
review?


From a cursory look at Tav's CPython patch, I'd describe it as follows:

Requires: an existing Python code execution environment
  which does not expose unsafe interpreter
  functionality to the (untrusted) code it runs,

Provides: a way for the trusted code outside of that
  restricted environment to wrap functions
  (which may reference trusted objects) in
  closures which can be passed into the restricted
  environment as completely opaque objects
  whose internal state (including any referenced
  trusted objects) cannot be accessed from the
  untrusted code.

When I say 'requires', I mean 'requires to be useful'; the patch can  
be applied to vanilla CPython, but isn't useful on its own.


Building blocks for a restricted execution environment as required by  
the patch are commonly provided in templating libraries; Tav mentions  
Genshi in particular. By default, Genshi templates don't come with  
security built in -- you can do what you please in a template.  
However, since Genshi manually constructs a Python AST from Python  
code in the template, one can restrict the AST and modify the builtins  
exposed to the template environment, making things like filesystem  
access, import and other sensitive system facilities unavailable.


Even with those unavailable, untrusted code can break out of  
containment by accessing object.__subclasses__ or gaining access to  
gi_frame and gi_code. This means you can't, for instance, pass into  
the untrusted code a closure which operates on trusted objects. Tav's  
patch addresses this particular problem.


To give you a complete sandbox to play with, I just violently ripped  
out the relevant code from Genshi, added some glue, and slapped it all  
together in a single file along with Tav's pure-Python code which is  
functionally equivalent to the CPython patch he submitted. The result,  
tested on 2.5.1 and nothing else:


http://radian.org/~krstic/sandbox.py

As is, sandbox allows you to execute untrusted Python code which won't  
have access to import, __import__, eval, exec, execfile, open, and to  
which you can pass closures which reference trusted objects which the  
untrusted code can't get at.


Example:

 import sandbox
 import sys
 import md5
 def trusted_pwd_closure(password):
...def get_pwd():
...return md5.md5(password).hexdigest()
...return get_pwd
...
 context = dict(pwd=trusted_pwd_closure('secret'))
 sandbox.run_string(print pwd(), context)
 sandbox.run_string(print pwd.func_closure[0].cell_contents,  
context)

[snip]
AttributeError: 'function' object has no attribute 'func_closure'

Without Tav's patch, the untrusted code can extract our password  
('secret') from the closure with that last statement.


To run whole files in the sandbox:
 sandbox.run_file('/some/path/file.py')

Finally, disclaimer: I put this together in *15 minutes*. I'm sure I  
missed something, this isn't secure, etc. It's just a proof of  
concept. Void where prohibited, etc.


Cheers,

--
Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] .pythonrc.py in man page

2009-02-22 Thread Mitchell L Model

The python man page, dated 2005 even in 3.1, has this curious entry:

  ~/.pythonrc.py
  User-specific initialization file loaded by the user module; not
  used by default or by most applications.

1. I couldn't figure out what the user module is.

2. I couldn't figure out what not used by default or by most 
applications meant: what would cause it to get loaded when python 
starts up? how would an application load it even if the user's 
environment didn't cause it to get loaded?


3. Why would this file exist if the environment variable 
PYTHONSTARTUP can specify a file to be loaded at startup?


Perhaps this entry in the man page is obsolete and should be removed?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] .pythonrc.py in man page

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 5:58 PM, Mitchell L Model mlmli...@comcast.net wrote:
 The python man page, dated 2005 even in 3.1, has this curious entry:

  ~/.pythonrc.py
  User-specific initialization file loaded by the user module;
 not
  used by default or by most applications.

 1. I couldn't figure out what the user module is.

user.py.

 2. I couldn't figure out what not used by default or by most applications
 meant: what would cause it to get loaded when python starts up? how would an
 application load it even if the user's environment didn't cause it to get
 loaded?

The docstring in user.py explains this.

 3. Why would this file exist if the environment variable PYTHONSTARTUP can
 specify a file to be loaded at startup?

PYTHONSTARTUP is only used in interactive sessions.

 Perhaps this entry in the man page is obsolete and should be removed?

Not at all.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] .pythonrc.py in man page

2009-02-22 Thread Daniel Stutzbach
On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum gu...@python.org wrote:

  Perhaps this entry in the man page is obsolete and should be removed?

 Not at all.


For what it's worth, the 2.6.1 documentation states: Deprecated since
version 2.6: The user module has been removed in Python 3.0.  If user.py
has indeed been removed, then Mitchell is correct.

http://docs.python.org/library/user.html

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Guido van Rossum
I'm not familiar with Genshi -- what is the purpose of the AST
transformation here?

Traditionally, sandboxing environments in Python usually just exec the
untrusted code in an environment with a __builtins__ dict that limits
the built-in functions and overrides __import__ so as to provide an
import implementation that allows import of pure-Python modules but
not extensions (or only allows certain extensions that have been
previously audited, or provides wrappers or subsets of selected
others).

--Guido

On Sun, Feb 22, 2009 at 6:09 PM, Ivan Krstić
krs...@solarsail.hcs.harvard.edu wrote:
 On Feb 22, 2009, at 5:15 PM, Martin v. Löwis wrote:

 What is the objective of this code? Is it a complete sandbox?
 If not, is a complete sandbox based on it available somehow for
 review?

 From a cursory look at Tav's CPython patch, I'd describe it as follows:

Requires: an existing Python code execution environment
  which does not expose unsafe interpreter
  functionality to the (untrusted) code it runs,

Provides: a way for the trusted code outside of that
  restricted environment to wrap functions
  (which may reference trusted objects) in
  closures which can be passed into the restricted
  environment as completely opaque objects
  whose internal state (including any referenced
  trusted objects) cannot be accessed from the
  untrusted code.

 When I say 'requires', I mean 'requires to be useful'; the patch can be
 applied to vanilla CPython, but isn't useful on its own.

 Building blocks for a restricted execution environment as required by the
 patch are commonly provided in templating libraries; Tav mentions Genshi in
 particular. By default, Genshi templates don't come with security built in
 -- you can do what you please in a template. However, since Genshi manually
 constructs a Python AST from Python code in the template, one can restrict
 the AST and modify the builtins exposed to the template environment, making
 things like filesystem access, import and other sensitive system facilities
 unavailable.

 Even with those unavailable, untrusted code can break out of containment by
 accessing object.__subclasses__ or gaining access to gi_frame and gi_code.
 This means you can't, for instance, pass into the untrusted code a closure
 which operates on trusted objects. Tav's patch addresses this particular
 problem.

 To give you a complete sandbox to play with, I just violently ripped out the
 relevant code from Genshi, added some glue, and slapped it all together in a
 single file along with Tav's pure-Python code which is functionally
 equivalent to the CPython patch he submitted. The result, tested on 2.5.1
 and nothing else:

http://radian.org/~krstic/sandbox.py

 As is, sandbox allows you to execute untrusted Python code which won't have
 access to import, __import__, eval, exec, execfile, open, and to which you
 can pass closures which reference trusted objects which the untrusted code
 can't get at.

 Example:

 import sandbox
 import sys
 import md5
 def trusted_pwd_closure(password):
 ...def get_pwd():
 ...return md5.md5(password).hexdigest()
 ...return get_pwd
 ...
 context = dict(pwd=trusted_pwd_closure('secret'))
 sandbox.run_string(print pwd(), context)
 sandbox.run_string(print pwd.func_closure[0].cell_contents, context)
 [snip]
 AttributeError: 'function' object has no attribute 'func_closure'

 Without Tav's patch, the untrusted code can extract our password ('secret')
 from the closure with that last statement.

 To run whole files in the sandbox:
 sandbox.run_file('/some/path/file.py')

 Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed
 something, this isn't secure, etc. It's just a proof of concept. Void where
 prohibited, etc.

 Cheers,

 --
 Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org





-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] .pythonrc.py in man page

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 6:30 PM, Daniel Stutzbach
dan...@stutzbachenterprises.com wrote:
 On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum gu...@python.org wrote:

  Perhaps this entry in the man page is obsolete and should be removed?

 Not at all.

 For what it's worth, the 2.6.1 documentation states: Deprecated since
 version 2.6: The user module has been removed in Python 3.0.  If user.py
 has indeed been removed, then Mitchell is correct.

 http://docs.python.org/library/user.html

Fair enough. I missed the 3.x context. Go ahead and fix the man page!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Ivan Krstić

On Feb 22, 2009, at 9:43 PM, Guido van Rossum wrote:

I'm not familiar with Genshi -- what is the purpose of the AST
transformation here?


Sorry, I should have been clearer. If the only goal is to provide a  
restricted bare interpreter, you can certainly just exec with a  
restricted set of builtins and no __import__. Since Tav mentioned  
wanting restricted execution of Genshi templates in particular (which  
have a rather complicated mechanism for executing inline Python code),  
I threw together a realistic, self-contained 'restricting Genshi' demo  
which doesn't rely on outside restrictions, such as those provided by  
GAE.


You can ignore the AST stuff; Genshi does it for its own (non- 
security) purposes.


--
Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread P.J. Eby

At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote:

I've received some enthusiastic emails from someone who wants to
revive restricted mode. He started out with a bunch of patches to the
CPython runtime using ctypes, which he attached to an App Engine bug:

http://code.google.com/p/googleappengine/issues/detail?id=671

Based on his code (the file secure.py is all you need, included in
secure.tar.gz) it seems he believes the only security leaks are
__subclasses__, gi_frame and gi_code. (I have since convinced him that
if we add restricted guards to these attributes, he doesn't need the
functions added to sys.)

I don't recall the exploits that Samuele once posted that caused the
death of rexec.py -- does anyone recall, or have a pointer to the
threads?


Just a question, but, if you just need a pure-python restricted 
environment for App Engine, why not just use the RestrictedPython 
package (i.e., http://pypi.python.org/pypi/RestrictedPython )?



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread Guido van Rossum
On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby p...@telecommunity.com wrote:
 At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote:

 I've received some enthusiastic emails from someone who wants to
 revive restricted mode. He started out with a bunch of patches to the
 CPython runtime using ctypes, which he attached to an App Engine bug:

 http://code.google.com/p/googleappengine/issues/detail?id=671

 Based on his code (the file secure.py is all you need, included in
 secure.tar.gz) it seems he believes the only security leaks are
 __subclasses__, gi_frame and gi_code. (I have since convinced him that
 if we add restricted guards to these attributes, he doesn't need the
 functions added to sys.)

 I don't recall the exploits that Samuele once posted that caused the
 death of rexec.py -- does anyone recall, or have a pointer to the
 threads?

 Just a question, but, if you just need a pure-python restricted environment
 for App Engine, why not just use the RestrictedPython package (i.e.,
 http://pypi.python.org/pypi/RestrictedPython )?

How does that work? Remember, app engine doesn't support certain
things, and bytecode manipulations (if that's what RestrictedPython
does) are one of the unsupported things.

The other reason I can think of is that Tav is a capabilities purist. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Reviving restricted mode?

2009-02-22 Thread P.J. Eby

At 07:56 PM 2/22/2009 -0800, Guido van Rossum wrote:

On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby p...@telecommunity.com wrote:
 Just a question, but, if you just need a pure-python restricted environment
 for App Engine, why not just use the RestrictedPython package (i.e.,
 http://pypi.python.org/pypi/RestrictedPython )?

How does that work? Remember, app engine doesn't support certain
things, and bytecode manipulations (if that's what RestrictedPython
does) are one of the unsupported things.


It doesn't modify bytecode, it modifies an AST.  It basically 
replaces prints, and attribute/item read/writes with function 
calls.  Unfortunately, it does this AST modification by running as a 
traversal against the stdlib compiler package's AST, not a modern 
AST.  So, I suppose it might not be usable as-is on app engine.


It does, however, have the advantage of having been used in Zope for 
oh, six or seven years now?  ISTM that it first came out around the 
same time as Python 2.3, and the latest version just dropped support 
for Python 2.1 and 2.2.  So, if you want something that wasn't thrown 
together in an afternoon, it might be a good thing to take a look at.  ;-)




The other reason I can think of is that Tav is a capabilities purist. :-)


You can implement capabilities on top of RestrictedPython; it's 
simply a policy-neutral enforcement framework.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Aahz
On Sun, Feb 22, 2009, Brett Cannon wrote:
 On Sat, Feb 21, 2009 at 20:12, Aahz a...@pythoncraft.com wrote:
 On Sat, Feb 21, 2009, Brett Cannon wrote:
 On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote:
 On Sat, Feb 21, 2009, Brett Cannon wrote:

 I am seeing two approaches emerging. One is where pickle contains all
 Python code and then uses something like use_extension to make sure
 the original Python objects are still reachable at some point. This
 has the drawback that you have to use some function to make the
 extensions happen and there is some extra object storage.

 The other approach is having pickle contain code known not to
 be overridden by anyone, import _pypickle for stuff that may be
 overridden, and then import _pickle for whatever is available. This
 approach has the perk of using a standard practice for how to pull in
 different implementation. But the drawback, thanks to how globals are
 bound, is that any code pulled in from _pickle/_pypickle will not be
 able to call into other optimized code; it's a take or leave it once
 the call chain enters one of those modules as they will always call
 the implementations in the module they originate from.

 To what extent do we care about being able to select Python-only on a
 per-module basis, particularly in the face of threaded imports?  That
 is,
 we could have a sys.python_only attribute that gets checked on import.
 That's simple and direct, and even allows per-module switching if the
 application really cares and import doesn't need to worry about threads.

 Alternatively, sys.python_only could be a set, but that gets ugly about
 setting from the application.  (The module checks to see whether it's
 listed in sys.python_only.)

 Maybe we should move this discussion to python-ideas for now to kick
 around really oddball suggestions?

 This is all about testing. If a change is made to some extension code it
 should be mirrored in the Python code and vice-versa.

 Okay, I don't see how that is a response to my suggestion -- I can
 imagine that someone might want to test a combination of pure-Python and
 binary libraries.
 
 I don't want to move it because this isn't some idea for a new feature that
 may or may not be useful; this isn't an idea, it's needed.

That's fine, but what about my idea of using sys.python_only?
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com