Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Stefan Behnel]

Tres Seaver wrote:
> Steven D'Aprano wrote:
>> Tres, for some reason every time you reply to the list, you send TWO 
>> copies instead of one:
> 
>> To: python-dev@python.org
>> CC: Python Dev 
> 
>> Could you please fix that?
> 
> I  can try:  I normally post via gmane, and leave python-dev CC'ed so
> that folks who read via the list don't have their replies to me fall off
> list (which happens on some lists, anyway).
> 
> I will trim the CC in the future, and hope for the best.

That's what works best for me, anyway.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Steven D'Aprano wrote:
> Tres, for some reason every time you reply to the list, you send TWO 
> copies instead of one:
> 
> To: python-dev@python.org
> CC: Python Dev 
> 
> Could you please fix that?

I  can try:  I normally post via gmane, and leave python-dev CC'ed so
that folks who read via the list don't have their replies to me fall off
list (which happens on some lists, anyway).

I will trim the CC in the future, and hope for the best.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuzWG+gerLs4ltQ4RAsQfAKCBGfeI6FEP8cNbOdh0zFhLjj65CgCgiLZb
725QgMYFCyhdM6OP5+SC04U=
=yRwI
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Steven D'Aprano]

Tres, for some reason every time you reply to the list, you send TWO 
copies instead of one:

To: python-dev@python.org
CC: Python Dev 

Could you please fix that?


-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> Network connectivity isn't a given, even today. So yes, there are
> environments that are secure (i.e. no network connectivity), and there
> are environments where developers are trusted (shock, horror) to
> actually follow company policy and get all licenses vetted by their
> Contracts group before installing downloaded software on company machines.
> 
> Given that even some of the core developers work in environments like
> that, then yes, I believe Python can and should take reasonable steps to
> enable its use in such situations.
> 
> And the most reasonably step Python can take on that front is to
> continue to provide a relatively powerful standard library *even if* a
> flexible and otherwise useful package management approach is added at
> some stage.

My inclination would be to leave the stdlib largely as is, except that
occostonally I would argue for ripping out a particular obsolete /
bitrotted module.

A couple of other points:

- - Absent a sufficiently powerful package management system, the pressure
  to add modules to the stdlib (or keep them) is higher because it is
  harder for *all* Python users to add them, or replace them if dropped.

- - The choice to add or remove a module to / from the stdlib should be
  made on the merits of the module, without regard to the kind of
  specialized deployment policies you outline.

- - Changing the stdlib in a new release of Python is probably irrelevant
  for the kind of environments you allude to, as there is likely as much
  review involved to approve a new version of Python as there was in
  approving it in the first place:  of the few I know of today, all are
  still running Python 2.1.x and / or 2.2.x for this reason.

> If someone else decides to create a MinimalPython which consists solely
> of something like easy_install and whatever is needed to run it (i.e.
> the opposite of the "fat" bundles from folks like ActiveState and
> Enthought), then more power to them. But I don't believe the official
> releases from python.org should go that way.

Note that I am *not* advocating scrubbing / exploding the stdlib.



Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuy9Y+gerLs4ltQ4RAranAJ4rCXgq0opHPki6OmlABbaqE3D1sQCeJ7Zt
Em6VMK1u+6+xYsoqixwfoJ4=
=YzN7
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: installation with ubuntu 8.04

[Jan Claeys]

Op vrijdag 13-03-2009 om 04:13 uur [tijdzone -0700], schreef John
Wagner:

> I am sorry to bother about this but I've asked for help from Canonical
> with no reply.

> They seem reluctant to update Python beyond version 2.5; I'm trying to
> install 3.0.1.

If you can wait until April 23rd then you will be able to use python 2.6
(as the default) or python 3.0.1 (as an option for your apps) on the
next release of Ubuntu (9.04).

Maybe Ubuntu will provide a package of python 3.0.1 in the 8.04
backports too, if there is enough demand for it.  And if not, somebody
can make a backport for Ubuntu 8.04 in a PPA on launchpad.


-- 
Jan Claeys

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Jan Claeys]

Op vrijdag 13-03-2009 om 12:28 uur [tijdzone +0300], schreef Oleg
Broytmann:
>Ext4 is not the only FS with delayed allocation.

Of course not, even ext3 has delayed allocation (even if 5 sec vs. 2 min
makes the disaster window a bit smaller).

The real problem seems to be that ext4 re-orders the rename (which it
does almost instantly) before the write (which waits for 2-15 minutes or
so).

There are other modern filesystems that take care such reordering
doesn't happen...


-- 
Jan Claeys

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Nick Coghlan]

Zvezdan Petkovic wrote:
> Of course, the above are C functions.  I don't think that Python
> programming is immune from such security considerations either.

The tempfile module exposes the same functionality (and uses mkstemp()
to create its filenames). It has also had features added over the years
to prevent automatic deletion of the temporary files, precisely so you
*can* grab them and rename them afterwards.

It actually wouldn't be a bad place to put a "create a temporary file
and rename it to  when closing it" helper class. Such a utility
could also include a way to request "fsync() before rename" behaviour
(off by default of course).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Nick Coghlan]

Tres Seaver wrote:
> You are plainly joking:  nothing in Python should know or care about the
> various bureaucratic insanities in some workplaces.  Given the
> *existing* stdlib and network connectivity, nothing any corporate
> security blackshirt can do will prevent an even moderately-motivated
> person from executing arbitrary code downloaded from elsewhere.  In that
> case, what is the point in trying to help those who impose such craziness?

Network connectivity isn't a given, even today. So yes, there are
environments that are secure (i.e. no network connectivity), and there
are environments where developers are trusted (shock, horror) to
actually follow company policy and get all licenses vetted by their
Contracts group before installing downloaded software on company machines.

Given that even some of the core developers work in environments like
that, then yes, I believe Python can and should take reasonable steps to
enable its use in such situations.

And the most reasonably step Python can take on that front is to
continue to provide a relatively powerful standard library *even if* a
flexible and otherwise useful package management approach is added at
some stage.

If someone else decides to create a MinimalPython which consists solely
of something like easy_install and whatever is needed to run it (i.e.
the opposite of the "fat" bundles from folks like ActiveState and
Enthought), then more power to them. But I don't believe the official
releases from python.org should go that way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Paul Moore]

2009/3/13 Tres Seaver :
> Paul Moore wrote:
>> 2009/3/13 Chris Withers :
>>> If a decent package management system *was* included, this wouldn't be an
>>> issue..
>>
>> Remember that a "decent package management system" needs to handle
>> filling in all the forms and arranging approvals to get authorisation
>> for packages when you download them.
>>
>> And no, I'm *not* joking. People in a locked-down corporate
>> environment really do benefit from just having to get the OK for
>> "Python", and then knowing that they have all they need.
>
> You are plainly joking:  nothing in Python should know or care about the
> various bureaucratic insanities in some workplaces.

I am not. What I *am* doing is saying (obliquely, I admit) is that for
a package management system to be "decent enough" for stripping down
the stdlib to not be an issue, it has to address these points (which
clearly it can't). In other words, the problems inherent in
restricting the stdlib aren't ones which are solvable by a package
management system.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Andrew McNabb]

On Fri, Mar 13, 2009 at 07:31:21PM +0100, "Martin v. Löwis" wrote:
> > Think about the security implications of a file name that is in advance
> > known to an attacker as well as the fact that the said file will replace
> > an *important* system file.
> 
> You should always use O_EXCL in that case. Relying on random name will
> be a severe security threat to the application.

But mkstemp does open files with O_EXCL, so the two approaches really
aren't that different.  Using tempfile can be a little simpler because
it will eventually succeed.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Zvezdan Petkovic]

On Mar 13, 2009, at 2:31 PM, Martin v. Löwis wrote:

Think about the security implications of a file name that is in  
advance known to an attacker as well as the fact that the said file  
will replace an *important* system file.


You should always use O_EXCL in that case. Relying on random name will
be a severe security threat to the application.


If you read an implementation of mkstemp() function, you'll see that  
it does exactly that:


if ((*doopen = open(path, O_CREAT|O_EXCL|O_RDWR, 0600)) >= 0)
return(1);
if (errno != EEXIST)
return(0);

That's why I mentioned mkstemp() in the OP.

Zvezdan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Martin v. Löwis]

> Think about the security implications of a file name that is in advance
> known to an attacker as well as the fact that the said file will replace
> an *important* system file.

You should always use O_EXCL in that case. Relying on random name will
be a severe security threat to the application.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Trent Mick]

Chris Withers wrote:

That said, it may make sense to just give greater prominence to
existing repackagers, such as ActiveState or Enthought.


Right, getting ActivePython or similar approved might be the way to go, 
but I'm betting it depends on the project...


Apologies for jumping in mid-thread here. FYI: We're (where "we" == 
ActiveState here) looking at spending more effort on Python of late. 
Some of our thoughts are on add modules: whether added to the 
ActivePython core or easily addable via an equivalent to ActivePerl's 
ppm (package manager) is still be batted around.


I'm curious as to people's thoughts. I'll also be at PyCon in Chicago 
getting thoughts.


Cheers,
Trent

--
Trent Mick
trentm at activestate.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

Chris Withers wrote:
> Jim Jewett wrote:
 - python to grow a decent, cross platform, package management system
>>
>> As stated, this may be impossible, because of the difference in what a
>> package should mean on Windows vs Unix.
>>
>> If you just mean a way to add python packages from pypi as with
>> EasyInstall, then maybe.
> 
> I meant package in the python sense, which has a clear definition,
> unless I'm mistaken?

Unfortunately, you are mistaken: the term "package" is highly confusing
and ambiguous in the python sense. It could be a
package-as-import-sees-it, or it could be a package-as-pypi-sees-it.
For the latter, the distutils inventors tried to coin the term
"distribution", but that didn't quite catch on. It *is* the Python
*Package* index, and people do often refer to the things it indexes
as packages.

>> In some environments, each new component must be approved.  Once
>> python is approved, the standard library is OK, but adding 7 packages
>> from pypi requires 7 more sets of approvals.
> 
> True, but as I mentioend elsewhere, I myself haven't done a python
> project where I only needed python and the standard lib for many years...

I was always able to get what I need through aptitude.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

>> I don't like the standard library to shrink. It's good that batteries
>> are included.
> 
> If a decent package management system *was* included, this wouldn't be
> an issue..

You can prove anything with a false premise... I believe that a package
management system that is decent cannot possibly be included in Python
(IOW, any packaging system included in Python cannot be decent enough
to allow removal of things from the standard library)

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (03/06/09 - 03/13/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2388 open (+38) / 14920 closed (+16) / 17308 total (+54)

Open issues with patches:   837

Average duration of open issues: 658 days.
Median duration of open issues: 399 days.

Open Issues Breakdown
   open  2359 (+37)
pending29 ( +1)

Issues Created Or Reopened (56)
___

2to3 wrong for types.StringTypes 03/08/09
   http://bugs.python.org/issue5425reopened hagen   
  
   patch   

cmpfunc in Python 3.0.1 windows installer03/10/09
   http://bugs.python.org/issue5431reopened Nigel Galloway  
  
   

plistlib contains unescaped hex sequence in doc string   03/06/09
   http://bugs.python.org/issue5432created  jim.correia 
  
   

Excessive optimization in IncrementalNewlineDecoder  03/06/09
CLOSED http://bugs.python.org/issue5433created  pitrou  
  
   patch   

datetime.MonthDelta  03/07/09
   http://bugs.python.org/issue5434created  jess.austin 
  
   patch   

test_httpservers on Debian Testing   03/07/09
   http://bugs.python.org/issue5435created  zamotcr 
  
   

test_distutils fails with official Mac OS X Installer Disk Image 03/07/09
   http://bugs.python.org/issue5436created  oefe
  
   

Singleton MemoryError can hold traceback data and locals indefin 03/08/09
   http://bugs.python.org/issue5437created  pitrou  
  
   patch   

test_bigmem.test_from_2G_generator uses more memory than expecte 03/08/09
   http://bugs.python.org/issue5438created  pitrou  
  
   

string.strip behaves strangly03/08/09
CLOSED http://bugs.python.org/issue5439created  dwjang  
  
   

string.strip behaves strangly03/08/09
CLOSED http://bugs.python.org/issue5440created  dwjang  
  
   

Convenience API for timeit.main  03/08/09
   http://bugs.python.org/issue5441created  ncoghlan
  
   easy

[3.1alpha1] test_importlib fails on Mac OSX 10.5.6   03/08/09
   http://bugs.python.org/issue5442created  cartman 
  
   

trivial typo in itertools documentation  03/10/09
   http://bugs.python.org/issue5443reopened donlorenzo  
  
   patch   

.chm build process on Windows doesn't use the right filename 03/08/09
   http://bugs.python.org/issue5444created  gagenellina 
  
   patch   

codecs.StreamWriter.writelines problem when passed generator 03/08/09
   http://bugs.python.org/issue5445created  dlesco  
  
   

doc building progress on Windows doesn't show any color  03/08/09
CLOSED http://bugs.python.org/issue5446created  gagenellina 
  
   patch   

future unicode literals and r'\u'03/08/09
   http://bugs.python.org/issue5447created  pooryorick  
  
   patch   

Add precision property to decimal.Decimal03/08/09
CLOSED http://bugs.python.org/issue5448created  dlesco  
  
 

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Lie Ryan wrote:

> Tres Seaver wrote:
>> I'm not arguing that employees should violate their employers' policies:
>>  I'm arguing that Python itself shouldn't try to cater to such policies.
> 
> Basically you're saying: Python is designed not to work on such environment.

No, I'm saying that it isn't Python's responibility to enable that kind
of policy.  If it happens to be good *for Python* to have a a package
installation / upgrade machinery (the real point of the discussion),
then it will be up to the paranoid admin to figure out how to disable
that feature:  it isn't the problem of the Python developers.

There are real costs to "batteries included," especially for modules
which don't get used much.  One such cost is that an unused module tends
to bitrot over time;  another is that the presence of a module in the
stdlib may "shadow" other, better modules / packages which are not in
the stdlib.  Those costs need to be balanced against the undoubted
benefits, when making the choice to add or remove a module from the stdlib.

>>  Note that I'm not talking about running code pushed on me by malware
>> authors, either:  I'm talking about "ordinary" software development
>> activities like using a script from a cookbook, or using a well-tested
>> and supported library, rather than NIH.
> 
> Some companies have /very/ strict policies on running anything on live 
> server, including scripts you write yourself. The problem is if the 
> script goes awry, it might disturb the stability or even security of the 
> server.

I understand that such policies exist, and why.  However, I don't think
their existence is a relevant constraint on the design or implementation
of Python.

>> Given that the out-of-the-box Python install already has facilities for
>> retrieving text over the net and executing that text, the notion of
>> "locking down" a machine to include only the bits installed in the stock
>> Python install is just "security theatre;"  such a machine shouldn't
>> have Python installed at all (nor a C compiler, etc.)
> 
> When the server administrator is already freaked out about adding an 
> script developed by in-house employee, what about adding an external module?

An admin who is that paranoid shouldn't be giving people shell access,
either:  given shell acccess, network connectivity, and the existing
stdlib the admin's policy is unenforceable as a technical measure.  Even
if the user can't create a file anywhere on the filesystem, the
interpreter prompt is enough to allow the user to evade the policy.
Heck, the *bash* prompt is enough to wreck it, e.g. using "here
documents," even without network connectivity.

As an aisde:  anybody who is installing packages from PyPI on a
production box (rather than using an index under their own control)
isn't sufficiently paranoid:  it can and does happen that people
re-upload changed packages to PyPI without changing the version, for
instance, not to mention removing older releases.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuooU+gerLs4ltQ4RAsdnAKCSkKc94bHvHBIrILampl9+Mksz8wCeJSBe
+Yl5HVmwQ6StGTcWNmDiSjE=
=qGID
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: installation with ubuntu 8.04

[Christian Heimes]

Steve Holden wrote:
> Look in the build instructions for "alt-install", and make sure that you
> *know* when you install the new Python that it isn't touching the system
> Python at all. Always run make -n install and check what the script will
> do before running it until you know what you are doing. Then run 3.x
> from (somewhere like) /usr/local/bin.

We anticipated the issue during the development cycle of py3k. The 3.x
series doesn't install a "python" command when it's installed with "make
install".

$ make install
[...]
* Note: not installed as 'python'.
* Use 'make fullinstall' to install as 'python'.
* However, 'make fullinstall' is discouraged,
* as it will clobber your Python 2.x installation.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Lie Ryan]

Tres Seaver wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Lie Ryan wrote:

Tres Seaver wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Paul Moore wrote:

2009/3/13 Chris Withers :

If a decent package management system *was* included, this wouldn't be an
issue..

Remember that a "decent package management system" needs to handle
filling in all the forms and arranging approvals to get authorisation
for packages when you download them.

And no, I'm *not* joking. People in a locked-down corporate
environment really do benefit from just having to get the OK for
"Python", and then knowing that they have all they need.

You are plainly joking:  nothing in Python should know or care about the
various bureaucratic insanities in some workplaces.  Given the
*existing* stdlib and network connectivity, nothing any corporate
security blackshirt can do will prevent an even moderately-motivated
person from executing arbitrary code downloaded from elsewhere.  In that
case, what is the point in trying to help those who impose such craziness?
I (and most people, I presume) would not run arbitrary program 
downloaded from somewhere else on a corporate server that holds many 
important customer data even when there is no technical or even 
bureaucratic restriction, maybe I will sneak around on a workstation but 
definitely not on the server especially if I love my job and want to 
keep it (I'm a student though so that applies to me in the future).


I'm not arguing that employees should violate their employers' policies:
 I'm arguing that Python itself shouldn't try to cater to such policies.


Basically you're saying: Python is designed not to work on such environment.


 Note that I'm not talking about running code pushed on me by malware
authors, either:  I'm talking about "ordinary" software development
activities like using a script from a cookbook, or using a well-tested
and supported library, rather than NIH.


Some companies have /very/ strict policies on running anything on live 
server, including scripts you write yourself. The problem is if the 
script goes awry, it might disturb the stability or even security of the 
server.



Given that the out-of-the-box Python install already has facilities for
retrieving text over the net and executing that text, the notion of
"locking down" a machine to include only the bits installed in the stock
Python install is just "security theatre;"  such a machine shouldn't
have Python installed at all (nor a C compiler, etc.)


When the server administrator is already freaked out about adding an 
script developed by in-house employee, what about adding an external module?


Of course all of this does not (usually) apply to regular workstation. A 
messed up workstation only means a reinstall, a messed up server may 
mean company reputation.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[Antoine Pitrou]

Aahz  pythoncraft.com> writes:
> 
> On Fri, Mar 13, 2009, Antoine Pitrou wrote:
> > R. David Murray  bitdance.com> writes:
> >> 
> >> You will note that what
> >> I suggested was that applications that _use the sync feature_ make
> >> it user controllable.
> > 
> > I'm sorry, but if it has nothing to do with Python itself, perhaps we
> > could stop this subthread (or move it to another ML)? There are enough
> > messages already.
> 
>   Yes, this *is* about Python: how should Python support David's
> use-case?

Python already has plenty of mechanisms for enabling configuration choices, so I
don't see your point: the use case is /already/ supported assuming there is a
sync() method at all.

The discussion is (or should be) about whether/how Python should expose a sync()
facility in its file input/output stack (or somewhere else in the stdlib).
Configuration policies are not for us to decide.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Oleg Broytmann]

On Fri, Mar 13, 2009 at 12:28:07PM +0300, Oleg Broytmann wrote:
> On Thu, Mar 12, 2009 at 10:14:41PM -0600, Adam Olsen wrote:
> > Yet the ext4
> > developers didn't see it that way, so it was sacrificed to new
> > performance improvements (delayed allocation).
> 
>Ext4 is not the only FS with delayed allocation. New XFS has it, btrfs
> will have it. Don't know about other OS/FS (ZFS? NTFS?)

http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/

   Ted Tso said HFS+ and ZFS have the property as well. So no, it is not
a deficiency in the Linux kernel or in Ext4 FS - it is a mainstream path in
modern filesystem design.

Oleg.
-- 
 Oleg Broytmannhttp://phd.pp.ru/p...@phd.pp.ru
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Lie Ryan wrote:
> Tres Seaver wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Paul Moore wrote:
>>> 2009/3/13 Chris Withers :
 If a decent package management system *was* included, this wouldn't be an
 issue..
>>> Remember that a "decent package management system" needs to handle
>>> filling in all the forms and arranging approvals to get authorisation
>>> for packages when you download them.
>>>
>>> And no, I'm *not* joking. People in a locked-down corporate
>>> environment really do benefit from just having to get the OK for
>>> "Python", and then knowing that they have all they need.
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> I (and most people, I presume) would not run arbitrary program 
> downloaded from somewhere else on a corporate server that holds many 
> important customer data even when there is no technical or even 
> bureaucratic restriction, maybe I will sneak around on a workstation but 
> definitely not on the server especially if I love my job and want to 
> keep it (I'm a student though so that applies to me in the future).

I'm not arguing that employees should violate their employers' policies:
 I'm arguing that Python itself shouldn't try to cater to such policies.
 Note that I'm not talking about running code pushed on me by malware
authors, either:  I'm talking about "ordinary" software development
activities like using a script from a cookbook, or using a well-tested
and supported library, rather than NIH.

Given that the out-of-the-box Python install already has facilities for
retrieving text over the net and executing that text, the notion of
"locking down" a machine to include only the bits installed in the stock
Python install is just "security theatre;"  such a machine shouldn't
have Python installed at all (nor a C compiler, etc.)



Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJunUx+gerLs4ltQ4RAojAAKCdoliiVDoGoKzfGXNuQUZVmoPrhgCfXeSa
pGCKI3wLt9W1A4ccnINSdLs=
=3H9u
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Lie Ryan]

Tres Seaver wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Paul Moore wrote:

2009/3/13 Chris Withers :

If a decent package management system *was* included, this wouldn't be an
issue..

Remember that a "decent package management system" needs to handle
filling in all the forms and arranging approvals to get authorisation
for packages when you download them.

And no, I'm *not* joking. People in a locked-down corporate
environment really do benefit from just having to get the OK for
"Python", and then knowing that they have all they need.


You are plainly joking:  nothing in Python should know or care about the
various bureaucratic insanities in some workplaces.  Given the
*existing* stdlib and network connectivity, nothing any corporate
security blackshirt can do will prevent an even moderately-motivated
person from executing arbitrary code downloaded from elsewhere.  In that
case, what is the point in trying to help those who impose such craziness?


I (and most people, I presume) would not run arbitrary program 
downloaded from somewhere else on a corporate server that holds many 
important customer data even when there is no technical or even 
bureaucratic restriction, maybe I will sneak around on a workstation but 
definitely not on the server especially if I love my job and want to 
keep it (I'm a student though so that applies to me in the future).


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[Aahz]

On Fri, Mar 13, 2009, Antoine Pitrou wrote:
> R. David Murray  bitdance.com> writes:
>> 
>> You will note that what
>> I suggested was that applications that _use the sync feature_ make
>> it user controllable.
> 
> I'm sorry, but if it has nothing to do with Python itself, perhaps we
> could stop this subthread (or move it to another ML)? There are enough
> messages already.

  Yes, this *is* about Python: how should Python support David's
use-case?  This discussion started because Python currently doesn't have
good mechansisms for fine-grained synching, and people are discussing how
Python should support various use-cases.  Please don't be too aggressive
about labeling discussion off-topic.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of 
indirection."  --Butler Lampson
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] py: urls, new bazaar plugin available

[Andrii V. Mishkovskyi]

On Thu, Mar 12, 2009 at 6:26 PM, Barry Warsaw  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hello Bazaar users!
>
> There's a new Bazaar plugin you can use to more easily access read-only or
> read-write branches on code.python.org.  This plugin provides the 'py:' url
> prefix.  For example, to get the trunk branch with the plugin installed, you
> can now do:
>
>    bzr branch py:trunk
>
> or to get the 2.6 branch you can do:
>
>    bzr branch py:2.6
>
> You can also use this to get user branches, e.g. my email rewrite branch:
>
>    bzr branch py:~barry/30email
>
> If you have write access to branches on code.python.org, you can either set
> the environment variable $PYDEV_USER or the Bazaar configuration option
> pydev_user (the value doesn't matter) to get bzr+ssh access instead of the
> standard http access.  py: works both for branching and pushing.
>
> Thanks to Karl Fogel for the implementation.  You'll need Karl's pydev
> plugin branch, and instructions on installing this are available here:
>
>    http://tinyurl.com/aq55oc
>
> I've updated the wiki page with additional details:
>
>    http://wiki.python.org/moin/Bazaar

And here is anouncement of similar Mercurial extension (slightly more
general, though):
http://selenic.com/pipermail/mercurial/2009-March/024547.html
Should this go to http://wiki.python.org/moin/Mercurial (this page
doesn't exist yet) or somewhere else?

>
> Enjoy!
> Barry
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSbk3xHEjvBPtnXfVAQIiVgQAt3GwmDSkFjog/mp4PxVKn/F6MQoEDa/A
> 0nNysiU2oEvUXDBWPlab2V53tqpZ/uvOS17hl7ZhlDe+Z2jUUYiCkH+Hfvpz5F9n
> u0Gf+5dMA7GQkLhvOezu7r6ngu2mmBB84ZwAfX4tJM+bBuQEn+U3BuRspkDiCb0S
> iZONBdHyk5g=
> =Pb2v
> -END PGP SIGNATURE-
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/mishok13%40gmail.com
>



-- 
--
Wbr, Andrii V. Mishkovskyi.

He's got a heart of a little child, and he keeps it in a jar on his desk.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Zvezdan Petkovic]

On Mar 12, 2009, at 3:15 PM, Martin v. Löwis wrote:

You still wouldn't use the tempfile module in that case. Instead, you
would create a regular file, with the name base on the name of the
important file.


If the file is *really* important, you actually want to use a  
temporary, randomly chosen, *unpredictable* name.


Think about the security implications of a file name that is in  
advance known to an attacker as well as the fact that the said file  
will replace an *important* system file.


See the details in any man page on mkstemp() and why it was introduced  
to replace a predictable mktemp().  Also notice that even mktemp() is  
better then what you proposed above.


Of course, the above are C functions.  I don't think that Python  
programming is immune from such security considerations either.


Zvezdan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Paul Moore wrote:
> 2009/3/13 Chris Withers :
>> If a decent package management system *was* included, this wouldn't be an
>> issue..
> 
> Remember that a "decent package management system" needs to handle
> filling in all the forms and arranging approvals to get authorisation
> for packages when you download them.
> 
> And no, I'm *not* joking. People in a locked-down corporate
> environment really do benefit from just having to get the OK for
> "Python", and then knowing that they have all they need.

You are plainly joking:  nothing in Python should know or care about the
various bureaucratic insanities in some workplaces.  Given the
*existing* stdlib and network connectivity, nothing any corporate
security blackshirt can do will prevent an even moderately-motivated
person from executing arbitrary code downloaded from elsewhere.  In that
case, what is the point in trying to help those who impose such craziness?

> Even ignoring the above,

Which the language and library should do.

> your "decent package management system" has
> to deal with systems with no internet connectivity - just copying the
> Python installer onto my pen drive to put on my Mum's laptop so I can
> write some admin jobs for her, is a lot easier than having to pick and
> choose from PyPI what to download before I start.

Nobody is arguing that there should be *no* batteries in the stdlib:  we
are talking about the ones which are leaking acid already, or which
might get there soon due to lack of maintenance.

> -1 on slimming down the stdlib.
> (OTOH, I've no problem with seeing an improved package system - just
> don't use it as an excuse to cripple the stdlib!)

Part of this discussion is about not *expanding* the stdlib:  give a
reasonable packaging story, leaving a given component out of the library
is a defensible choice.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJumAJ+gerLs4ltQ4RApC7AKDVsIxfBlw6CWWLa+VhaASyDz+LFQCfQp5I
yzrdYPo1FbXGAB90Ucf/Le8=
=bCTx
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: installation with ubuntu 8.04

[John Wagner]

Nick:

Thanks, man, for even replying.  I'm ashamed for posting such a stupid
question, and of course, you're right, as I should have known.  I'm used to
answering people's silly questions about c, and am an utter newb in python;
I'm trying to learn Scheme at the same time, so it's kind of hectic around
here.  My question is a newbie question, though, and I may as well face it.
I appreciate the polite tone of your reply.  If I can ever do anything to
help, I'll remember your example.

Regards,
John

On Fri, Mar 13, 2009 at 5:58 AM, Nick Coghlan  wrote:

> John,
>
> I'm afraid the auto-response to the python-3000 list is a little
> misleading. That list was for the initial development of python 3.0, and
> hence the redirect was to this list which is to discuss the
> *development* of the Python interpreter (both 2.x and 3.x).
>
> For more general questions (such as installation), the appropriate place
> to ask is python-l...@python.org (also available as the newsgroup
> comp.lang.python).
>
> Regards,
> Nick.
>
> P.S.
>
> > This mailing list is closed now. Please use python-dev@python.org
> > 
> > instead.
>
> The mailman powers-that-be may want to update the python-3000
> auto-response to direct people to python-list for normal questions and
> python-dev for development specific questions.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ---
>



-- 
John Wagner
Los Angeles
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Steve Holden]

R. David Murray wrote:
> On Fri, 13 Mar 2009 at 09:58, Chris Withers wrote:
>> Martin v. L�wis wrote:
>>> >  In light of this, what I'd love to see (but sadly can't really help
>>> >  with, and am not optimistic about happening) is for:
>>> > >  - python to grow a decent, cross platform, package management
>>> system
>>> > >  - the standard library to actually shrink to a point where only
>>> >  libraries that are not released elsewhere are included
>>> > >  I'd be interested to know how many users of python also felt
>>> this way >  ;-)
>>>
>>>  I don't like the standard library to shrink. It's good that batteries
>>>  are included.
>>
>> If a decent package management system *was* included, this wouldn't be
>> an issue..
> 
> I disagree.  One of the jobs I've had is release management for
> internal software projects that depend on various external pieces.
> Release integration tested against specific versions of those external
> packages, and those were the packages that needed to wind up on the system
> when the release was installed.  I've done systems depending on both perl
> and python, and let me tell you, python is way, _way_ easier to manage.
> With python, I have a dependency on a particular python version, and then
> maybe one or two add on packages.  With perl, I have perl, and then I
> have a gadzillion cpan modules.  I don't know how many a gadzillion is,
> because what I wound up doing was making a local copy of the cpan archive,
> checking that in to the repository, and writing up some scripts that made
> sure I pulled the actual install from my cpan snapshot and supported the
> developers in updating that snapshot when we were building a new version.
> (Nor was that the only problem with perlwhat idiot decided it was
> OK to interactively prompt for things during a batch install process?!
> And without providing any way to script the answers, at least that I
> could find!)
> 
> So I'm +1 for keeping the Python stdlib as comprehensive as sensible.
> (Please note that last word...I've no objection to pruning things
> that are no longer serving a useful purpose, or that are better
> managed outside the core.)
> 
Just for clarity, when I said a "jumbo distribution" I meant one with
all necessary libraries to run and support a specified set of python
functionalities. The point is *not* to add other libraries (which
invites version creep and undermines stability) but to have everything
you need for a given (core plus) set of functionality.

I believe my original message acknowledged that this wouldn't be
everyone's cup of tea, but if a "good"* set of applications were
analyzed and a distribution built to support just those (presumably
Python) subsystems, you would have a good core that you can augment with
the system-installed Python if you are so minded.

A jumbo shouldn't try to replace the system-installed Python because
hopefully different Python (jumbo) distributions would coexist on the
same system, supporting those Python elements for which their
configuration is acceptable. I would not expect core developers to have
to give the jumbos much mind, except in so far as they might require
support for (slightly?) different versions of Python.

A 1.5.2 process can talk to a 3.1 process without any problems at all as
long as the application protocol is unambiguous. Why this insistence on
trying to do everything with a single interpreter? Sure, it might use
more resources to have three different versions of PIL in use from three
different jumbos, but let's cross that bridge when we come to it.

Naturally, in Python there are already several environments that will
compute the required library subset necessary to support an application,
though at present they do it only across a single Python version and
application. However, writing a simple GUI or command-line program to
call the other Python modules will give you a single analyzable module
tree. You don't even have to distribute the GUI if you don't want ...

So I don't see "jumbo" as replacing "batteries included". More like
"comes with 14v 300AH accumulator and support for domain name and email
services" or "suitable for GeoDjango virtuals under VirtualBox with NAT
addressing". For non-Python stuff (e.g. PostgreSQL, though SQLite is
plenty good enough for experiments) I think the API has to be stable
enough to accommodate some version variations.

regards
 Steve

* This is not a democracy: use your own prejudices to decide.
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Formatting mini-language suggestion

[Stephen J. Turnbull]

Greg Ewing writes:

 > I don't think you'll want to code the separators into
 > all your format strings in that case, either. You'll
 > want some sort of context that you set up for the
 > page you're about to serve.

Sure.  But the POSIX locale is not a good solution, nor is it a
building block for such a solution.  If this PEP *can't* provide such
a building block, too bad, life is like that.  If it can, but we don't
bother to look for it because of locale, for shame!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: installation with ubuntu 8.04

[Steve Holden]

John:

First, you have stumbled on to the wring list. Generally a list whose
name ends in -dev is for the devs of the project. Hence this is the list
where the Python devs live. In future posts like this should be sent to
python-list (comp.lang.python).

Having said that, Canonical's "reluctance" to support Python 3.x is
(IMHO) simply good engineering caution on their part. If you were to
(accidentally) replace the system Python with 3.x you would undoubtedly
break Ubuntu, which is very Python-oriented, so have a care!

Look in the build instructions for "alt-install", and make sure that you
*know* when you install the new Python that it isn't touching the system
Python at all. Always run make -n install and check what the script will
do before running it until you know what you are doing. Then run 3.x
from (somewhere like) /usr/local/bin.

You may find for a while that there's much less support code available
for the 3.x version. The transition to 3.x is really only just beginning.

regards
 Steve

John Wagner wrote:
> 
> 
> -- Forwarded message --
> From: **  >
> Date: Thu, Mar 12, 2009 at 8:32 PM
> Subject: installation with ubuntu 8.04
> To: gnuj...@gmail.com 
> 
> 
> This mailing list is closed now. Please use python-dev@python.org
> 
> instead.
> 
> 
> 
> -- Forwarded message --
> From: John Wagner mailto:gnuj...@gmail.com>>
> To: python-3...@python.org 
> Date: Thu, 12 Mar 2009 20:32:55 -0700
> Subject: installation with ubuntu 8.04
> I am sorry to bother about this but I've asked for help from Canonical
> with no reply.
> They seem reluctant to update Python beyond version 2.5; I'm trying to
> install 3.0.1.
> I'm a newb, and have tried rtfm--does someone have the incantation required?
> Thanks v. much in advance.
> 
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: installation with ubuntu 8.04

[Nick Coghlan]

John,

I'm afraid the auto-response to the python-3000 list is a little
misleading. That list was for the initial development of python 3.0, and
hence the redirect was to this list which is to discuss the
*development* of the Python interpreter (both 2.x and 3.x).

For more general questions (such as installation), the appropriate place
to ask is python-l...@python.org (also available as the newsgroup
comp.lang.python).

Regards,
Nick.

P.S.

> This mailing list is closed now. Please use python-dev@python.org
> 
> instead.

The mailman powers-that-be may want to update the python-3000
auto-response to direct people to python-list for normal questions and
python-dev for development specific questions.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[R. David Murray]

On Fri, 13 Mar 2009 at 12:27, Antoine Pitrou wrote:

R. David Murray  bitdance.com> writes:


You will note that what
I suggested was that applications that _use the sync feature_ make
it user controllable.


I'm sorry, but if it has nothing to do with Python itself, perhaps we could stop
this subthread (or move it to another ML)? There are enough messages already.


It has to do with python in that it has to do with how this proposed
new feature gets documented.  But I agree, we've gone on too long
about it already.  I'll make a note to myself to bring this up again
(only if warranted) when the feature actually lands and I can see how
it is documented.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: installation with ubuntu 8.04

[John Wagner]

-- Forwarded message --
From: 
Date: Thu, Mar 12, 2009 at 8:32 PM
Subject: installation with ubuntu 8.04
To: gnuj...@gmail.com


This mailing list is closed now. Please use python-dev@python.org
instead.



-- Forwarded message --
From: John Wagner 
To: python-3...@python.org
Date: Thu, 12 Mar 2009 20:32:55 -0700
Subject: installation with ubuntu 8.04
I am sorry to bother about this but I've asked for help from Canonical with
no reply.
They seem reluctant to update Python beyond version 2.5; I'm trying to
install 3.0.1.
I'm a newb, and have tried rtfm--does someone have the incantation required?
Thanks v. much in advance.

-- 
John Wagner
Los Angeles





-- 
John Wagner
Los Angeles
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[Antoine Pitrou]

R. David Murray  bitdance.com> writes:
> 
> You will note that what
> I suggested was that applications that _use the sync feature_ make
> it user controllable.

I'm sorry, but if it has nothing to do with Python itself, perhaps we could stop
this subthread (or move it to another ML)? There are enough messages already.

Thanks

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[R. David Murray]

On Fri, 13 Mar 2009 at 14:27, Steven D'Aprano wrote:

On Fri, 13 Mar 2009 01:02:26 pm R. David Murray wrote:

On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:

R. David Murray  bitdance.com> writes:

Seriously, though, the point is that IMO an application should not
be calling fsync unless it provides a way for that behavior to be
controlled by the user.


But whether an application does it or not is none of Python's
business, is it? What is the disagreement exactly?


I'd like to see whatever feature gets added support the application
writer in making this user controllable, or at the very least
document that this to do so is best practice if you use the sync
feature.


It's not best practice. It may be best practice for a certain class of
users and applications, e.g. those who value the ability to control
low-level behaviour of the app, but it is poor practice for other
classes of users and applications. Do you really think that having
Minefield make the file syncing behaviour of the high scores file
user-configurable is best practice? People care about their high
scores, but they don't care that much.


Why would Minefield bother to use sync/fsync?  You will note that what
I suggested was that applications that _use the sync feature_ make
it user controllable.  And yes, it is Best Practice, because we have
glaring counterexamples (sqlite3, the one mentioned by another poster)
that require people to jump through hoops to compensate for the fact
that it isn't user controllable.

I thought that a significant majority of applications wouldn't have
to care.

If most applications do really have to care (ie: there's no way for them
to recover gracefully from a crash that trashes the new data they were
trying to save unless they call fsync), then my argument is a lot weaker.
And from the post that talked about the problems with rename, I gather
I was misunderstanding the extent of the problem.

Perhaps I can restrict my request: that it be noted that if you use fsync
_aggressively_ (ie: not just on a final save that happens infrequently)
that you make it user controllable in some fashion.  Note that that
user control could be as simple as being able to set the autosave
interval on an autosaving editor if the autosave needs to do an fsync
for safety.

Thinking about it further, I think what I'm really looking for is a
warning to applications developers that using fsync can have significant
performance penalties (and therefore if they use it a lot they should
make it configurable), and can make laptop users hate you if it gets
used any time other than at a user requested save point :)

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Rough draft: Proposed format specifier for a thousands separator (discussion moved from python-dev)

[Nick Coghlan]

Eric Smith wrote:
> I like approach 2 in general. I'll give some thought to other, similar
> schemes which would allow "8," or "8,d" to work. I think people will
> write "8," and expect "   1,234", not an error.

Given that PEP 3101 deliberately exposed the underlying Formatter class
to support more flexible expansions to formatting, I think trying to
build that level of flexibility into the formatting mini-language itself
would be a mistake.

So step 1 would be to add the "," or "'" to the format string to
indicate the use of separators. While both violate the rule of syntax
not looking like grit on Tim's monitor, the latter has the virtues of
not only being harder to confuse with a period, but also being
consistent with the printf() grouping flag as defined in the Single Unix
Specification v2 (as noted by JYK earlier in the thread and described in
more detail at [1]).

Then step 2 (if deemed appropriate) would be to expose the mini-language
parser in a way that makes it easier to write a custom format_field()
method in a Formatter subclass without having to completely duplicate
the existing parser's functionality.

Step 1 makes sense even if step 2 is never taken. Step 2 also makes
sense even if step 1 is never taken. So I don't think the fact that step
2 is probably a good idea should be used to hold up PEP 378, nor do I
think PEP 378 should be complicated by locale related issues that are
better handled by making it easier to write a custom format_field()
method in a Formatter subclass.

Cheers,
Nick.

[1] http://linux.die.net/man/3/printf

P.S. This post is really just an elaboration of this comment from JYK:

"""I'm not against Raymond's proposal, just against doing a *bad* job of
making it work in multiple locales. Locale conventions can be complex,
and are going to be best represented outside the format string."""


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[R. David Murray]

On Fri, 13 Mar 2009 at 09:58, Chris Withers wrote:

Martin v. L?wis wrote:

>  In light of this, what I'd love to see (but sadly can't really help
>  with, and am not optimistic about happening) is for:
> 
>  - python to grow a decent, cross platform, package management system
> 
>  - the standard library to actually shrink to a point where only

>  libraries that are not released elsewhere are included
> 
>  I'd be interested to know how many users of python also felt this way 
>  ;-)


 I don't like the standard library to shrink. It's good that batteries
 are included.


If a decent package management system *was* included, this wouldn't be an 
issue..


I disagree.  One of the jobs I've had is release management for
internal software projects that depend on various external pieces.
Release integration tested against specific versions of those external
packages, and those were the packages that needed to wind up on the system
when the release was installed.  I've done systems depending on both perl
and python, and let me tell you, python is way, _way_ easier to manage.
With python, I have a dependency on a particular python version, and then
maybe one or two add on packages.  With perl, I have perl, and then I
have a gadzillion cpan modules.  I don't know how many a gadzillion is,
because what I wound up doing was making a local copy of the cpan archive,
checking that in to the repository, and writing up some scripts that made
sure I pulled the actual install from my cpan snapshot and supported the
developers in updating that snapshot when we were building a new version.
(Nor was that the only problem with perlwhat idiot decided it was
OK to interactively prompt for things during a batch install process?!
And without providing any way to script the answers, at least that I
could find!)

So I'm +1 for keeping the Python stdlib as comprehensive as sensible.
(Please note that last word...I've no objection to pruning things
that are no longer serving a useful purpose, or that are better
managed outside the core.)

--
R. David Murray   http://www.bitdance.com___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Nick Coghlan]

Martin v. Löwis wrote:
>> auto-delete is one of the nice features of tempfile.  Another feature
>> which is entirely appropriate to this usage, though, though, is creation
>> of a non-conflicting filename.
> 
> Ok. In that use case, however, it is completely irrelevant whether the
> tempfile module calls fsync. After it has generated the non-conflicting
> filename, it's done.

I agree, but my comment was that it would be nice if better fsync
support (if added) could be done in such a way that it helped not only
file objects, but also *file-like* objects (such as the wrappers in the
tempfile module) without making the file-like API any fatter.

If that's not possible or practical so be it, but it is still something
to keep in mind when considering options.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Paul Moore]

2009/3/13 Chris Withers :
> If a decent package management system *was* included, this wouldn't be an
> issue..

Remember that a "decent package management system" needs to handle
filling in all the forms and arranging approvals to get authorisation
for packages when you download them.

And no, I'm *not* joking. People in a locked-down corporate
environment really do benefit from just having to get the OK for
"Python", and then knowing that they have all they need.

Even ignoring the above, your "decent package management system" has
to deal with systems with no internet connectivity - just copying the
Python installer onto my pen drive to put on my Mum's laptop so I can
write some admin jobs for her, is a lot easier than having to pick and
choose from PyPI what to download before I start.

-1 on slimming down the stdlib.
(OTOH, I've no problem with seeing an improved package system - just
don't use it as an excuse to cripple the stdlib!)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Chris Withers]

Jim Jewett wrote:

- python to grow a decent, cross platform, package management system


As stated, this may be impossible, because of the difference in what a
package should mean on Windows vs Unix.

If you just mean a way to add python packages from pypi as with
EasyInstall, then maybe.


I meant package in the python sense, which has a clear definition, 
unless I'm mistaken?



In some environments, each new component must be approved.  Once
python is approved, the standard library is OK, but adding 7 packages
from pypi requires 7 more sets of approvals.


True, but as I mentioend elsewhere, I myself haven't done a python 
project where I only needed python and the standard lib for many years...



That said, it may make sense to just give greater prominence to
existing repackagers, such as ActiveState or Enthought.


Right, getting ActivePython or similar approved might be the way to go, 
but I'm betting it depends on the project...


Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Formatting mini-language suggestion

[Greg Ewing]

Stephen J. Turnbull wrote:

Greg Ewing writes:
 > If an app is using a particular thousands separator in
 > one place, it will probably want to use it everywhere.

Not if that app is internationalized (eg, a webapp that serves both
Americans and Chinese).


I don't think you'll want to code the separators into
all your format strings in that case, either. You'll
want some sort of context that you set up for the
page you're about to serve.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Chris Withers]

Michael Foord wrote:
I have mixed feelings. It is great that the batteries are included, but 
some batteries are showing their age or not maintained (who maintains 
IDLE? - does the calendar module really warrant being in the standard 
library? - imaplib is really not useful and IMAPClient which isn't in 
the standard library is much better [1]).


Wow, interesting case in point ;-)
I actually stumbled into imaplib, found it not useful and gave up and 
solved the problem a completely different way. Had I not tripped over 
it, I might have found IMAPClient!


Chris
--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Chris Withers]

Steve Holden wrote:

Perhaps we could encourage more "jumbo" distributions, like Enthought's
and ActiveState's. I suspect many people would rather be able to
maintain their Python functionality as a single product.


I think you'll find it split..

People who use and love things like zc.buildout do so because they want 
to free package maintainers to do their own release cycles and not have 
individual packages held back by the need to release the "whole project" 
in one go.


However, yes, I'm sure there are just as many people who want to install 
"just one thing" and have it all there. (although they'll be sadly 
disappointed when they realise whatever it is they need (lxml, PIL, 
xlrd,xlwt) isn't there.


That said, a decent package management system in the core *and* the 
jumbo installers you mention would likely keep both camps happy ;-)


Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Chris Withers]

Martin v. Löwis wrote:

In light of this, what I'd love to see (but sadly can't really help
with, and am not optimistic about happening) is for:

- python to grow a decent, cross platform, package management system

- the standard library to actually shrink to a point where only
libraries that are not released elsewhere are included

I'd be interested to know how many users of python also felt this way ;-)


I don't like the standard library to shrink. It's good that batteries
are included.


If a decent package management system *was* included, this wouldn't be 
an issue..


Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Ext4 data loss

[Oleg Broytmann]

On Thu, Mar 12, 2009 at 10:14:41PM -0600, Adam Olsen wrote:
> Yet the ext4
> developers didn't see it that way, so it was sacrificed to new
> performance improvements (delayed allocation).

   Ext4 is not the only FS with delayed allocation. New XFS has it, btrfs
will have it. Don't know about other OS/FS (ZFS? NTFS?)

Oleg.
-- 
 Oleg Broytmannhttp://phd.pp.ru/p...@phd.pp.ru
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss

[Lie Ryan]

Steven D'Aprano wrote:

On Fri, 13 Mar 2009 01:02:26 pm R. David Murray wrote:

On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:

R. David Murray  bitdance.com> writes:

Seriously, though, the point is that IMO an application should not
be calling fsync unless it provides a way for that behavior to be
controlled by the user.

But whether an application does it or not is none of Python's
business, is it? What is the disagreement exactly?

I'd like to see whatever feature gets added support the application
writer in making this user controllable, or at the very least
document that this to do so is best practice if you use the sync
feature.


It's not best practice. It may be best practice for a certain class of 
users and applications, e.g. those who value the ability to control 
low-level behaviour of the app, but it is poor practice for other 
classes of users and applications. Do you really think that having 
Minefield make the file syncing behaviour of the high scores file 
user-configurable is best practice? People care about their high 
scores, but they don't care that much.


It may even lead to more data loss than leaving it out:

* If the application chooses a specific strategy, this strategy might 
(for the sake of the argument) lead to data loss once in ten million 
writes on average.


* If the application makes this a configuration option, the increased 
complexity of writing the code, and the increased number of paths that 
need to be tested, may lead to bugs which cause data loss. This may be 
more risky than the original strategy above (whatever that happens to 
be.)


Complexity is not cost-free, and insisting that the more complex, 
expensive solution is always "best practice" is wrong.


If the pops and moms uses a financial program and lost their only copy 
of 10 years worth of financial data, they'll simply be confused and 
that's it.


Meanwhile if a network administrator needs to squeeze the last bit of 
performance out of his backup script, he definitely would threaten the 
dev-team of the programming language to make manual sync file writing 
the default, since it makes it difficult for him to fine-tune the 
syncing process.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Formatting mini-language suggestion

[Stephen J. Turnbull]

Greg Ewing writes:
 > Nick Coghlan wrote:
 > 
 > >   [[fill]align][sign][#][0][minimumwidth][,sep][.precision][type]
 > > 
 > > 'sep' is the new field that defines the thousands separator.
 > 
 > Wouldn't it be better to use a locale setting for this,
 > instead of having to specify it in every format string?

Maybe, but the POSIX locale concept is broken (because it's process-
global) in many contexts.  Viz.

 > If an app is using a particular thousands separator in
 > one place, it will probably want to use it everywhere.

Not if that app is internationalized (eg, a webapp that serves both
Americans and Chinese).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com