Re: [Python-Dev] Issue #11051: system calls per import

[Guido van Rossum]

On Sun, Jan 30, 2011 at 11:33 PM, "Martin v. Löwis"  wrote:
>> Maybe also
>>
>>    * Read and cache the directory contents and search it ourselves
>>      instead of making a system call for every possible name.
>
> I wouldn't do that - I would expect that this is actually slower than
> making the system calls, because the system might get away with not
> reading the entire directory (whereas it will have to when we explicitly
> ask for that).

Hm. Long (very long) ago I had to implement just that, and it was much
faster. But this was over NFS. Still, I think the directory would have
to be truly enormous before reading its contents (which doesn't access
all the inodes) is slower than statting a few dozen of its entries. At
least on most *nix filesystems.

Another thing to consider: on App Engine (which despite of all its
architectural weirdness uses a -- mostly -- standard Linux filesystem
for the Python code of the app) someone measured that importing from a
zipfile is much faster than importing from the filesystem. I would
imagine this extends to other contexts too, and it makes sense because
the zipfile directory gets cached in memory so no stat() calls are
necessary.

(Basically I am biased to believe that stat() is a pretty slow system
call -- this may just be old NFS lore though.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Byte code arguments from two to one byte: did anyone try this?

[Jurjen N.E. Bos]

I tried to find any research on this subject, but I couldn't find any,
so I'll be daring and vulnerable and just try it out to see what your  
thoughts

are.
I single stepped a simple loop in Python to see where the efficiency  
bottlenecks are.
I was impressed by the optimizations already in there, but I still  
dare to suggest an optimization that from my estimates might shave  
off a few cycles, speeding up Python about 5%.
The idea is simple: change the byte code argument values from two  
bytes to one.

Implications are:
- code changes are relatively simple, see below
- fewer memory reads, which are becoming more and more expensive
- saves three instructions for every opcode with args (i.e. most of  
them)



Code changes are, as far as I could find:
compile.c:
assemble_emit must produce extended opcodes
for all cases of more than 8 bits instead of 16

ceval.c:
NEXTARG and PEEKARG need adjustment
EXTENDED_ARG needs adjustment
(this will be a four byte instruction, which is ugly, I agree)

peephole.c:
GETARG, SETARG, need adjustment
also GETJUMPTGT, CODESIZE
routine tuple_of_constants, fold_binops_on_constants, PyCode_Optimize
are dependent on instruction length, which will be 2 instead of 3
(search for the digit 3 will find all cases, as far as I checked)
you probably will have to write a macro for codestr[i+3]
there is a check for code length >32700, but I think this one might  
stay,

maybe if a few extra checks are added.

dis:
minor adjustments


Estimation of speed impact:
about 80% of the instructions seem to have an argument, and I never  
saw an  opcode >255 while looking at bytecode, so they are probably  
not frequent.


The NEXTARG macro expands on my Macbook to:

mov-408(%ebp),%edx(next_instr)
movzbl 2(%edx),%eax   (*second byte)
shl$0x8,%eax  (*shift)
movzbl 1(%edx),%edx   (first byte)
add%edx,%eax  (*combine)

and the starred instructions will vanish.
The main loop is approximately 40 instructions, so a saving of three  
instructions is significant. I don't dare to claim 3/40 = 7.5% savings,

but I think 5% may be realistic.

Did anyone try this already? If not, I might take up the gauntlet
and try it myself, but I never did this before...


- Jurjen

PS I also saw that some scratch variables, mainly v and x, are  
carefull stored back in memory by the compiler and the end of the big  
interpreter loop, while their value isn't used anymore, of course.
A few carefully placed braces might tell the compiler how useless  
this is and

save another few percent.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Byte code arguments from two to one byte: did anyone try this?

[Stefan Behnel]

Jurjen N.E. Bos, 31.01.2011 10:17:

I single stepped a simple loop in Python to see where the efficiency
bottlenecks are.


What version of CPython did you try that with? The latest py3k branch?

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [RELEASED] Python 3.2 rc 2

[Georg Brandl]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On behalf of the Python development team, I'm quite happy to announce
the second release candidate of Python 3.2.

Python 3.2 is a continuation of the efforts to improve and stabilize the
Python 3.x line.  Since the final release of Python 2.7, the 2.x line
will only receive bugfixes, and new features are developed for 3.x only.

Since PEP 3003, the Moratorium on Language Changes, is in effect, there
are no changes in Python's syntax and built-in types in Python 3.2.
Development efforts concentrated on the standard library and support for
porting code to Python 3.  Highlights are:

* numerous improvements to the unittest module
* PEP 3147, support for .pyc repository directories
* PEP 3149, support for version tagged dynamic libraries
* PEP 3148, a new futures library for concurrent programming
* PEP 384, a stable ABI for extension modules
* PEP 391, dictionary-based logging configuration
* an overhauled GIL implementation that reduces contention
* an extended email package that handles bytes messages
* a much improved ssl module with support for SSL contexts and certificate
  hostname matching
* a sysconfig module to access configuration information
* additions to the shutil module, among them archive file support
* many enhancements to configparser, among them mapping protocol support
* improvements to pdb, the Python debugger
* countless fixes regarding bytes/string issues; among them full support
  for a bytes environment (filenames, environment variables)
* many consistency and behavior fixes for numeric operations

For a more extensive list of changes in 3.2, see

http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2 visit:

http://www.python.org/download/releases/3.2/

Please consider trying Python 3.2 with your code and reporting any bugs
you may notice to:

http://bugs.python.org/


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk1Gj6IACgkQN9GcIYhpnLC53wCfcZhc6bxbc+fsmi+PAJxM6npr
Hh4An3QRdeyKHm+L3CqVk+EX02PxNx2r
=sTu6
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Byte code arguments from two to one byte: did anyone try this?

[Steven D'Aprano]

Jurjen N.E. Bos wrote:
I was impressed by the optimizations already in there, but I still dare 
to suggest an optimization that from my estimates might shave off a few 
cycles, speeding up Python about 5%.
The idea is simple: change the byte code argument values from two bytes 
to one.



Interesting. Have you seem Cesare Di Mauro's WPython project, which 
takes the opposite strategy?


http://code.google.com/p/wpython2/



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #11051: system calls per import

[Jussi Enkovaara]

On 2011-01-30 21:43, "Martin v. Löwis" wrote:

Am 30.01.2011 17:54, schrieb Alexander Belopolsky:

On Sun, Jan 30, 2011 at 11:35 AM, Victor Stinner
  wrote:
..

We should find a compromise between speed (limit the number of system
calls) and the usability of Python modules.


Do you have measurements that show python spending significant time on
failing open calls?


No; past measurements always showed that this is insignificant, probably
thanks to operating system caching the relevant directory blocks (so
it doesn't really matter whether you make one or ten lookups per
directory; my guess is that it matters more if you look into ten
directories instead of one).


Dear Python-developers,
I would like you to be aware of one particular problem related to the system calls 
in massively parallel systems. We are developing a Python-based simulation software 
GPAW (https://wiki.fysik.dtu.dk/gpaw/) and tested it with up to tens of thousands 
of CPU cores. The program uses MPI, thus thousands of Python interpreters are 
launched at start-up time. As all these interpreters execute the same import 
statements, the huge amount of (IO-related) system calls puts extreme pressure to 
the file system, and as result just starting the Python interpreter(s) can take ~45 
minutes with ~30 000 CPU cores!


Currently, we have tried to work around the problem either by installing Python and 
required additional modules (NumPy and GPAW) to a ramdisk, or by modifying the 
CPython source (at the moment 2.6 version) in such a way that only single process 
performs the system calls and uses MPI to broadcast the results to other processes 
(preliminary work in progress).


As a related problem, dynamic linking can also be quite expensive (or even not 
available in some systems), and in some cases we have made a small hack to CPython 
for enabling statically linked packages (simple modules can of course be included 
relatively easily in static Python build.)


I am not expecting that the problems can be solved easily for the general CPython 
interpreter, especially as massively parallel supercomputers are quite small niche 
of Python usage. However, I think it would be good to be aware of problems with 
large amount of system calls in a more special Python usage.


Best regards,
Jussi
--
Jussi Enkovaara, Application Scientist, High Performance Computing, CSC
PO. BOX 405 02101 Espoo, Finland, Tel +358 9 457 2935, fax +358 9 457 2302
CSC - IT Center for Science, www.csc.fi, e-mail: jussi.enkova...@csc.fi
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Followup: Byte code arguments from two to one byte: did anyone try this?

[Jurjen N.E. Bos]

What version of CPython did you try that with? The latest py3k branch?


I had a quick look at 3.2, 2.5 and 2.7 and got the impression that  
the savings is  more if the interpreter loop is faster: the fewer  
instructions there are, the bigger a 3 instruction difference would  
make.


The NEXTARG macro is the same in all three versions:
#define NEXTARG()   (next_instr += 2, (next_instr[-1]<<8) +  
next_instr[-2])

and the compiler compiles this to two separate fetches.

I found out my compiler (gcc) will make better code if we used a short.
It produces a "movswl" instruction to do both fetches at the same  
time, if I force it to.

That saves two instructions already.

This would imply that on little-endian machines, this would already  
save a few percent changing just 1 line of code in ceval.c:

#define NEXTARG()   (next_instr += 2, *(short *)&next_instr[-2])

- Jurjen
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] short fetch for NEXTARG macro (was: one byte byte code arguments)

[Jurjen N.E. Bos]

I just did it: my first python source code hack.
I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to  
short pointer, and lo and behold, a crude measurement indicates one  
to two percent speed increase.

That isn't much, but it is virtually for free!

Here are the macro's I used:
#define NEXTARG() (next_instr +=2, *(short*)&next_instr[-2])
#define PEEKARG() (*(short*)&next_instr[1])

- Jurjen
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #11051: system calls per import

[Antoine Pitrou]

On Mon, 31 Jan 2011 00:08:25 -0800
Guido van Rossum  wrote:
> 
> (Basically I am biased to believe that stat() is a pretty slow system
> call -- this may just be old NFS lore though.)

I don't know about NFS, but starting a Python interpreter located on a
Samba share from a Windows VM is quite slow too.
I think Martin is right for the common case: on a local filesystem on a
modern Unix, stat() is certainly very fast. Remote or
distributed filesystems seem to be more of a problem.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] short fetch for NEXTARG macro (was: one byte byte code arguments)

[Antoine Pitrou]

On Mon, 31 Jan 2011 13:28:39 +0100
"Jurjen N.E. Bos"  wrote:
> I just did it: my first python source code hack.
> I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to  
> short pointer, and lo and behold, a crude measurement indicates one  
> to two percent speed increase.
> That isn't much, but it is virtually for free!
> 
> Here are the macro's I used:
> #define NEXTARG() (next_instr +=2, *(short*)&next_instr[-2])
> #define PEEKARG() (*(short*)&next_instr[1])

Some architectures forbid unaligned access, so this can't be used as-is.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] short fetch for NEXTARG macro (was: one byte byte code arguments)

[Cesare Di Mauro]

2011/1/31 Antoine Pitrou 

> On Mon, 31 Jan 2011 13:28:39 +0100
> "Jurjen N.E. Bos"  wrote:
> > I just did it: my first python source code hack.
> > I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to
> > short pointer, and lo and behold, a crude measurement indicates one
> > to two percent speed increase.
> > That isn't much, but it is virtually for free!
> >
> > Here are the macro's I used:
> > #define NEXTARG() (next_instr +=2, *(short*)&next_instr[-2])
> > #define PEEKARG() (*(short*)&next_instr[1])
>
> Some architectures forbid unaligned access, so this can't be used as-is.
>
> Regards
>
> Antoine.
>
>
WPython already addressed it (
http://code.google.com/p/wpython2/source/browse/Python/ceval.c?repo=wpython11):

#ifdef WORDS_BIGENDIAN
#define NEXTOPCODE() oparg = *next_instr++; \
opcode = oparg >> 8; oparg &= 0xff
#else
#define NEXTOPCODE() oparg = *next_instr++; \
opcode = oparg & 0xff; oparg >>= 8
#endif

Shorts alignament is also guaranted due to wordcodes (
http://wpython2.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdfpag.12).

Cesare
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Byte code arguments from two to one byte: did anyone try this?

[Terry Reedy]

On 1/31/2011 5:31 AM, Steven D'Aprano wrote:

Jurjen N.E. Bos wrote:

I was impressed by the optimizations already in there, but I still
dare to suggest an optimization that from my estimates might shave off
a few cycles, speeding up Python about 5%.
The idea is simple: change the byte code argument values from two
bytes to one.



Interesting. Have you seem Cesare Di Mauro's WPython project, which
takes the opposite strategy?

http://code.google.com/p/wpython2/


The two strategies could be mixed. Some 'word codes' could consist of a 
bytecode + byte arg, and others a real word code. Maybe WPython does 
that already. Might end up being slower though.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Byte code arguments from two to one byte: did anyone try this?

[Cesare Di Mauro]

2011/1/31 Terry Reedy 

> On 1/31/2011 5:31 AM, Steven D'Aprano wrote:
>
>> Jurjen N.E. Bos wrote:
>>
>>> I was impressed by the optimizations already in there, but I still
>>> dare to suggest an optimization that from my estimates might shave off
>>> a few cycles, speeding up Python about 5%.
>>> The idea is simple: change the byte code argument values from two
>>> bytes to one.
>>>
>>
>>
>> Interesting. Have you seem Cesare Di Mauro's WPython project, which
>> takes the opposite strategy?
>>
>> http://code.google.com/p/wpython2/
>>
>
> The two strategies could be mixed. Some 'word codes' could consist of a
> bytecode + byte arg, and others a real word code. Maybe WPython does that
> already. Might end up being slower though.
>
> --
>  Terry Jan Reedy


Yes, WPython already does it (
http://wpython2.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdfpag.7)
, but on average it was faster (pag. 28).

Cesare

>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] short fetch for NEXTARG macro (was: one byte byte code arguments)

[James Y Knight]

On Jan 31, 2011, at 7:45 AM, Antoine Pitrou wrote:

> On Mon, 31 Jan 2011 13:28:39 +0100
> "Jurjen N.E. Bos"  wrote:
>> I just did it: my first python source code hack.
>> I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to  
>> short pointer, and lo and behold, a crude measurement indicates one  
>> to two percent speed increase.
>> That isn't much, but it is virtually for free!
>> 
>> Here are the macro's I used:
>> #define NEXTARG() (next_instr +=2, *(short*)&next_instr[-2])
>> #define PEEKARG() (*(short*)&next_instr[1])
> 
> Some architectures forbid unaligned access, so this can't be used as-is.

It could perhaps be #ifdef'd in on x86/x86-64, though, which is by far the most 
common architecture to run python on.

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #11051: system calls per import

[Barry Warsaw]

On Jan 30, 2011, at 05:35 PM, Victor Stinner wrote:

>And the real question is: should we change that before 3.2 final? If we
>don't change that in 3.2, it will be harder to change it later (but it
>is still possible).

I don't see how you possibly can without re-entering beta.  Mucking with the
import machinery *at all* does not seem prudent in the last RC. ;)

FWIW, I recall this being discussed at the time of the PEPs and we decided not
to narrow the search patterns down.  I'd have to go through my archives for
the details, but I think it would be better to officially deprecate the
'module' form so that they can be removed in a future version.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[techtonik]

Reviewers: ,



Please review this at http://codereview.appspot.com/4080047/

Affected files:
  M Tools/msi/msi.py
  M Tools/msi/msilib.py


Index: Tools/msi/msi.py
===
--- Tools/msi/msi.py(revision 88279)
+++ Tools/msi/msi.py(working copy)
@@ -4,7 +4,6 @@
 import msilib, schema, sequence, os, glob, time, re, shutil, zipfile
 from msilib import Feature, CAB, Directory, Dialog, Binary, add_data
 import uisample
-from win32com.client import constants
 from distutils.spawn import find_executable
 from uuids import product_codes
 import tempfile
@@ -1360,7 +1359,7 @@

 # Step 2: Add CAB files
 i = msilib.MakeInstaller()
-db = i.OpenDatabase(msi, constants.msiOpenDatabaseModeTransact)
+db = i.OpenDatabase(msi, msilib.msiOpenDatabaseModeTransact)

 v = db.OpenView("SELECT LastSequence FROM Media")
 v.Execute(None)
Index: Tools/msi/msilib.py
===
--- Tools/msi/msilib.py (revision 88279)
+++ Tools/msi/msilib.py (working copy)
@@ -4,7 +4,6 @@
 import win32com.client.gencache
 import win32com.client
 import pythoncom, pywintypes
-from win32com.client import constants
 import re, string, os, sets, glob, subprocess, sys, _winreg, struct

 try:
@@ -29,6 +28,18 @@
 knownbits = datasizemask | type_valid | type_localizable | \
 typemask | type_nullable | type_key

+# Constants from Windows Installer SDK
+msiOpenDatabaseModeReadOnly = 0
+msiOpenDatabaseModeTransact = 1
+msiOpenDatabaseModeDirect = 2
+msiOpenDatabaseModeCreate = 3
+msiColumnInfoNames = 0
+msiColumnInfoTypes = 1
+msiReadStreamInteger = 0
+msiReadStreamBytes = 1
+msiViewModifyInsert = 1
+msidbFileAttributesVital = 512
+
 # Summary Info Property IDs
 PID_CODEPAGE=1
 PID_TITLE=2
@@ -141,8 +152,7 @@

 def gen_schema(destpath, schemapath):
 d = MakeInstaller()
-schema = d.OpenDatabase(schemapath,
-win32com.client.constants.msiOpenDatabaseModeReadOnly)
+schema = d.OpenDatabase(schemapath, msiOpenDatabaseModeReadOnly)

 # XXX ORBER BY
 v=schema.OpenView("SELECT * FROM _Columns")
@@ -196,8 +206,7 @@
 def gen_sequence(destpath, msipath):
 dir = os.path.dirname(destpath)
 d = MakeInstaller()
-seqmsi = d.OpenDatabase(msipath,
-win32com.client.constants.msiOpenDatabaseModeReadOnly)
+seqmsi = d.OpenDatabase(msipath, msiOpenDatabaseModeReadOnly)

 v = seqmsi.OpenView("SELECT * FROM _Tables");
 v.Execute(None)
@@ -212,7 +221,7 @@
 f.write("%s = [\n" % table)
 v1 = seqmsi.OpenView("SELECT * FROM `%s`" % table)
 v1.Execute(None)
-info = v1.ColumnInfo(constants.msiColumnInfoTypes)
+info = v1.ColumnInfo(msiColumnInfoTypes)
 while 1:
 r = v1.Fetch()
 if not r:break
@@ -226,7 +235,7 @@
 rec.append(r.StringData(i))
 elif info.StringData(i)[0]=="v":
 size = r.DataSize(i)
-bytes = r.ReadStream(i, size,  
constants.msiReadStreamBytes)

+bytes = r.ReadStream(i, size, msiReadStreamBytes)
 bytes = bytes.encode("latin-1") # binary data  
represented "as-is"

 if table == "Binary":
 fname = rec[0]+".bin"
@@ -275,7 +284,7 @@
 r.SetStream(i+1, field.name)
 else:
 raise TypeError, "Unsupported type %s" %  
field.__class__.__name__

-v.Modify(win32com.client.constants.msiViewModifyInsert, r)
+v.Modify(msiViewModifyInsert, r)
 r.ClearData()
 v.Close()

@@ -298,8 +307,7 @@
 ProductCode = ProductCode.upper()
 d = MakeInstaller()
 # Create the database
-db = d.OpenDatabase(name,
- win32com.client.constants.msiOpenDatabaseModeCreate)
+db = d.OpenDatabase(name, msiOpenDatabaseModeCreate)
 # Create the tables
 for t in schema.tables:
 t.create(db)
@@ -538,7 +546,7 @@
 short = self.make_short(file)
 full = "%s|%s" % (short, file)
 filesize = os.stat(absolute).st_size
-# constants.msidbFileAttributesVital
+# msidbFileAttributesVital
 # Compressed omitted, since it is the database default
 # could add r/o, system, hidden
 attributes = 512


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #11051: system calls per import

[Brett Cannon]

On Mon, Jan 31, 2011 at 04:43, Antoine Pitrou  wrote:
> On Mon, 31 Jan 2011 00:08:25 -0800
> Guido van Rossum  wrote:
>>
>> (Basically I am biased to believe that stat() is a pretty slow system
>> call -- this may just be old NFS lore though.)
>
> I don't know about NFS, but starting a Python interpreter located on a
> Samba share from a Windows VM is quite slow too.
> I think Martin is right for the common case: on a local filesystem on a
> modern Unix, stat() is certainly very fast. Remote or
> distributed filesystems seem to be more of a problem.

I should mention that I have considered implementing a caching finder
and loader for filesystems in importlib for people to optionally
install to use for themselves. The real trick, though, is should it
only cache hits, misses, or both? Regardless, though, it would be a
very simple mixin or subclass to implement if there is demand for this
sort of thing.

And as for the zipfile being faster, that's true (I have incomplete
benchmarks in importlib that you can use if people want to measure
this stuff themselves, although you will need to tweak them to run
against a zipfile).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Amaury Forgeot d'Arc]

Hi,

2011/1/31  :
> Reviewers: ,
>
> Please review this at http://codereview.appspot.com/4080047/
[...]

It looks good, but did you create an item in the issue tracker?

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[georg . brandl]

Is there a bugs.python.org issue for this?

http://codereview.appspot.com/4080047/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[techtonik]

There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch
to clean up the code a bit while I am trying to understand how to add
Python to the PATH.

I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
more beneficial to development as it doesn't require switching from
console to browser for submitting changes. This way tiny changes can be
integrated/updated more rapidly.

1.
http://mercurial.selenic.com/wiki/ContributingChanges#The_basics:_patches_by_email


http://codereview.appspot.com/4080047/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Brian Curtin]

On Mon, Jan 31, 2011 at 14:45,  wrote:

> There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch
> to clean up the code a bit while I am trying to understand how to add
> Python to the PATH.
>
> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
> more beneficial to development as it doesn't require switching from
> console to browser for submitting changes. This way tiny changes can be
> integrated/updated more rapidly.
>
> 1.
>
> http://mercurial.selenic.com/wiki/ContributingChanges#The_basics:_patches_by_email
>
>
> http://codereview.appspot.com/4080047/


Please create an issue.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Antoine Pitrou]

On Mon, 31 Jan 2011 20:45:45 +
techto...@gmail.com wrote:
> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
> more beneficial to development as it doesn't require switching from
> console to browser for submitting changes.

Ok, why don't you contribute to Mercurial instead?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Georg Brandl]

Am 31.01.2011 21:45, schrieb techto...@gmail.com:
> There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch
> to clean up the code a bit while I am trying to understand how to add
> Python to the PATH.
> 
> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
> more beneficial to development as it doesn't require switching from
> console to browser for submitting changes. This way tiny changes can be
> integrated/updated more rapidly.

The tracker is not bureaucracy, it's how our development process works.
I know that Mercurial uses a different process, with patches always going
to the mailing list and being reviewed there, but that would be way too
much volume for python-dev considering our number of patches.

BTW, you should be able to send emails to rep...@bugs.python.org in order
to create new issues, and attachments will automatically become attached
to the bug reports.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Ethan Furman]

techto...@gmail.com wrote:

I see no reason for b.p.o bureaucracy.


It provides a place for discussion, and makes it easier to coordinate 
multiple efforts.


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[anatoly techtonik]

On Fri, Jan 28, 2011 at 10:34 PM, Christian Heimes  wrote:
> Am 28.01.2011 20:29, schrieb Raymond Hettinger:
>> At the very least, we should add some prominent instructions for getting the 
>> command line version up and running.
>
> /me pops out of Guido's time machine and says: "execute
> Tools/scripts/win_add2path.py"
>
> I'm -1 on adding Python to %PATH%. The private MSVCRT DLLs may lead to
> unexpected side effects and it doesn't scale at all.

Can you explain that part? There are no any MSVCRT DLLs in my
Python26+ installation directories.

> What about people
> with more than one Python installation? I suggest that we add a single
> user specific directory or a global directory to %PATH% for all
> installations. Then the Python installer or 3rd party modules can drop
> executables like python3.3.exe or plip-3.3.exe into this directory.

python33.exe, but user story about people with more than one Python
installation is a different one.

> A
> .bat file won't do good because .bat files must be called with "call
> python33.bat" from another .bat file or the first one gets terminated.

Wow. I've spent so many years in Windows console and didn't know that. Thanks.

> We can even use a single and simple executable as template for all tasks:
>
>  * get registry key from resource section of the executable
>  * use the registry key to lookup the location and name of pythonXX.dll
>  * load DLL
>  * get optional dotted module name for resource section
>  * either fire up interpreter as shell, with **argv or -m
> dotted.module.name **argv
>
> Done ;)

Actually, I would like to see the code that dynamically finds
pythonXX.dll that is available on the system, and loads it into
memory. This will be extremely useful for writing 3rd party
application plugins in Python. Plugins that they only work when Python
is installed and it doesn't really matter which Python version is
there. But that is another story also.
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[anatoly techtonik]

Ok. Here is the patch. I used Orca to reverse installer tables of
Mercurial MSI and inserted similar entry for Python.

Also available for review at: http://codereview.appspot.com/4023055
-- 
anatoly t.
Index: Tools/msi/msi.py
===
--- Tools/msi/msi.py(revision 88279)
+++ Tools/msi/msi.py(working copy)
@@ -463,6 +463,11 @@
  ("CompileGrammar", "COMPILEALL", 6802),
 ])
 
+# Add target dir to PATH
+add_data(db, "Environment",
+[("Environmnent", "=-*PATH", "[~];[TARGETDIR]", "python.exe"), 
+])
+
 #
 # Standard dialogs: FatalError, UserExit, ExitDialog
 fatal=PyDialog(db, "FatalError", x, y, w, h, modal, title,
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Brian Curtin]

On Mon, Jan 31, 2011 at 15:13, anatoly techtonik wrote:

> Ok. Here is the patch. I used Orca to reverse installer tables of
> Mercurial MSI and inserted similar entry for Python.
>
> Also available for review at: http://codereview.appspot.com/4023055
> --
> anatoly t.


That's the easy part. It doesn't cover any of the real issues with doing
this.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[anatoly techtonik]

On Mon, Jan 31, 2011 at 11:24 PM, Brian Curtin  wrote:
> On Mon, Jan 31, 2011 at 15:13, anatoly techtonik 
> wrote:
>>
>> Ok. Here is the patch. I used Orca to reverse installer tables of
>> Mercurial MSI and inserted similar entry for Python.
>>
>> Also available for review at: http://codereview.appspot.com/4023055
>> --
>> anatoly t.
>
> That's the easy part. It doesn't cover any of the real issues with doing
> this.

Please be more specific. It will also help if you integrate this part
while it's still hot.
--
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Brian Curtin]

On Mon, Jan 31, 2011 at 15:43, anatoly techtonik wrote:

> On Mon, Jan 31, 2011 at 11:24 PM, Brian Curtin 
> wrote:
> > On Mon, Jan 31, 2011 at 15:13, anatoly techtonik 
> > wrote:
> >>
> >> Ok. Here is the patch. I used Orca to reverse installer tables of
> >> Mercurial MSI and inserted similar entry for Python.
> >>
> >> Also available for review at: http://codereview.appspot.com/4023055
> >> --
> >> anatoly t.
> >
> > That's the easy part. It doesn't cover any of the real issues with doing
> > this.
>
> Please be more specific. It will also help if you integrate this part
> while it's still hot.
> --
> anatoly t.
>

There are numerous comments in the various PATH-related issues on the issue
tracker, and many of them are duplicated in this very thread.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[anatoly techtonik]

On Mon, Jan 31, 2011 at 10:54 PM, Antoine Pitrou  wrote:
> On Mon, 31 Jan 2011 20:45:45 +
> techto...@gmail.com wrote:
>> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
>> more beneficial to development as it doesn't require switching from
>> console to browser for submitting changes.
>
> Ok, why don't you contribute to Mercurial instead?

If you don't want to receive a stupid answer, why don't you read the
link and say what you don't like in this approach in a constructive
manner?

http://mercurial.selenic.com/wiki/ContributingChanges#The_basics:_patches_by_email
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[anatoly techtonik]

On Mon, Jan 31, 2011 at 11:09 PM, Ethan Furman  wrote:
> techto...@gmail.com wrote:
>>
>> I see no reason for b.p.o bureaucracy.
>
> It provides a place for discussion, and makes it easier to coordinate
> multiple efforts.

Code review system provides a better space for discussion if we are
speaking about simple code cleanup. To me polluting tracker with the
issues that are neither bugs nor feature requests only makes bug
triaging process and search more cumbersome.
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[anatoly techtonik]

On Mon, Jan 31, 2011 at 10:58 PM, Georg Brandl  wrote:
> Am 31.01.2011 21:45, schrieb techto...@gmail.com:
>> There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch
>> to clean up the code a bit while I am trying to understand how to add
>> Python to the PATH.
>>
>> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
>> more beneficial to development as it doesn't require switching from
>> console to browser for submitting changes. This way tiny changes can be
>> integrated/updated more rapidly.
>
> The tracker is not bureaucracy, it's how our development process works.

Don't you want to improve this process? Code review system is a much
better place to review patches than mailing list or bug tracker.
Especially patches that are not related to actual bugs.

> I know that Mercurial uses a different process, with patches always going
> to the mailing list and being reviewed there, but that would be way too
> much volume for python-dev considering our number of patches.

Seems reasonable. Do you have any stats how many patches are sent
weekly and how many of them are actually integrated?

> BTW, you should be able to send emails to rep...@bugs.python.org in order
> to create new issues, and attachments will automatically become attached
> to the bug reports.

Thanks. I'll keep this in mind.
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[Brian Curtin]

On Mon, Jan 31, 2011 at 15:50, anatoly techtonik wrote:

> On Mon, Jan 31, 2011 at 10:54 PM, Antoine Pitrou 
> wrote:
> > On Mon, 31 Jan 2011 20:45:45 +
> > techto...@gmail.com wrote:
> >> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
> >> more beneficial to development as it doesn't require switching from
> >> console to browser for submitting changes.
> >
> > Ok, why don't you contribute to Mercurial instead?
>
> If you don't want to receive a stupid answer, why don't you read the
> link and say what you don't like in this approach in a constructive
> manner?
>
>
> http://mercurial.selenic.com/wiki/ContributingChanges#The_basics:_patches_by_email
> --
> anatoly t.


>>> Don't send your patch to the 
>>> BugTracker -
it can't be reviewed there, so it won't go anywhere!

We do fine with reviews on the tracker, and there has been some on and off
work on integrating Rietveld. For the people actually doing the work here,
accepting patches on the tracker and dealing with them there has been a
reasonably effective workflow, enough that we don't see a need to change it.

>>> Patches go to 
>>> mercurial-de...@selenic.com 
>>> - no
subscription necessary!

As you were directed to in an earlier email by Georg, there is now a way to
report bugs via email without requiring any subscription. *report*@*bugs*.*
python*.*org is the address.*
*
*
*>>> *Because this is a community project and our developers are very busy,
patches will sometimes fall through the cracks. If you've gotten no
response to your patch after a few days, feel free to resend it.

This is true of any workflow on just about any open source project. Whether
it's email or a bug tracker, not everything is going to be acknowledged,
reviewed, fixed, or rejected immediately. We feel that the tracker allows us
to, well, keep track of things. It works for us.


What they do works for them, and I'm sure it works great. Could it work for
python-dev? Maybe. Is it worth changing anything when no one who is doing
the actual work has voiced a need for change? Absolutely not.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[Antoine Pitrou]

On Mon, 31 Jan 2011 23:50:18 +0200
anatoly techtonik  wrote:
> On Mon, Jan 31, 2011 at 10:54 PM, Antoine Pitrou  wrote:
> > On Mon, 31 Jan 2011 20:45:45 +
> > techto...@gmail.com wrote:
> >> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
> >> more beneficial to development as it doesn't require switching from
> >> console to browser for submitting changes.
> >
> > Ok, why don't you contribute to Mercurial instead?
> 
> If you don't want to receive a stupid answer, why don't you read the
> link and say what you don't like in this approach in a constructive
> manner?

Very simple: I don't want to be spammed with tons of patches, patch
reviews, and issue comments. Also, I want the history of issue
discussions to be easily accessible from permanent, issue-specific
URLs, rather than search through mailing-list archives to understand
why a change was made.

I appreciate that you refrained from giving a stupid answer, however.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #11051: system calls per import

[Martin v. Löwis]

> Another thing to consider: on App Engine (which despite of all its
> architectural weirdness uses a -- mostly -- standard Linux filesystem
> for the Python code of the app) someone measured that importing from a
> zipfile is much faster than importing from the filesystem. I would
> imagine this extends to other contexts too, and it makes sense because
> the zipfile directory gets cached in memory so no stat() calls are
> necessary.

Of course, you can't know until you measure, and then you only know
about the specific case.

However, I think you can't really compare zip reading with directory
reading - I'd expect that reading a zip directory is signficantly faster
than reading the directory contents of the zip file unpacked, just
because this is so many fewer layers of indirection.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[Benjamin Peterson]

2011/1/31 anatoly techtonik :
> On Mon, Jan 31, 2011 at 10:54 PM, Antoine Pitrou  wrote:
>> On Mon, 31 Jan 2011 20:45:45 +
>> techto...@gmail.com wrote:
>>> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
>>> more beneficial to development as it doesn't require switching from
>>> console to browser for submitting changes.
>>
>> Ok, why don't you contribute to Mercurial instead?
>
> If you don't want to receive a stupid answer, why don't you read the
> link and say what you don't like in this approach in a constructive
> manner?

As I understand it, there used to be patc...@python.org. I'm not sure
why this was discontinued, so perhaps someone more senior should chime
in. :)



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Benjamin Peterson]

2011/1/31 anatoly techtonik :
> On Mon, Jan 31, 2011 at 11:09 PM, Ethan Furman  wrote:
>> techto...@gmail.com wrote:
>>>
>>> I see no reason for b.p.o bureaucracy.
>>
>> It provides a place for discussion, and makes it easier to coordinate
>> multiple efforts.
>
> Code review system provides a better space for discussion if we are
> speaking about simple code cleanup. To me polluting tracker with the
> issues that are neither bugs nor feature requests only makes bug
> triaging process and search more cumbersome.

If it's not a bug or a feature request, why does it need to change?



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Ethan Furman]

Brian Curtin wrote:
On Mon, Jan 31, 2011 at 15:43, anatoly techtonik > wrote:

That's the easy part. It doesn't cover any of the real issues with doing
this.


Please be more specific. It will also help if you integrate this part
while it's still hot.
--
anatoly t.


There are numerous comments in the various PATH-related issues on the 
issue tracker, and many of them are duplicated in this very thread.


Imagine that -- discussion and coordination on b.p.o.!  ;)

~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Nick Coghlan]

On Tue, Feb 1, 2011 at 7:58 AM, anatoly techtonik  wrote:
> To me polluting tracker with the
> issues that are neither bugs nor feature requests only makes bug
> triaging process and search more cumbersome.

Anatoly, your constant efforts to try to force python-dev to adapt to
*your* way of doing things, instead of being willing to work with the
documented processes are *seriously* annoying. Which is a shame, since
it obscures the fact that your underlying suggestions are often quite
reasonable.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[martin]

What's the rationale of this change? It certainly doesn't remove the
dependency from win32com.client (i.e. the code continues to depend on
win32com).



http://codereview.appspot.com/4080047/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finally fix installer to add Python to %PATH% on Windows

[Martin v. Löwis]

Am 31.01.2011 22:13, schrieb anatoly techtonik:
> Ok. Here is the patch. I used Orca to reverse installer tables of
> Mercurial MSI and inserted similar entry for Python.

This doesn't do uninstallation correctly.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[Martin v. Löwis]

>> If you don't want to receive a stupid answer, why don't you read the
>> link and say what you don't like in this approach in a constructive
>> manner?
> 
> As I understand it, there used to be patc...@python.org. I'm not sure
> why this was discontinued, so perhaps someone more senior should chime
> in. :)

As a mailing list, it was unmaintainable, since there was no tracking
of what patches still need consideration. So a web-based bug tracker
got into use (although I forgot the name of the tracker software that
was used before SourceForge).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial style patch submission (Was: MSI: Remove dependency from win32com.client module (issue4080047))

[Dirkjan Ochtman]

On Mon, Jan 31, 2011 at 22:50, anatoly techtonik  wrote:
> If you don't want to receive a stupid answer, why don't you read the
> link and say what you don't like in this approach in a constructive
> manner?

Mercurial is a much smaller project, so it has different needs. It
would be nice if you could respect the process the developers on any
project have laid out for their project and assume they know what
works best for them.

Cheers,

Dirkjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[techtonik]

It removes the dependency from msi.py making it easier to do the rest in
subsequent patches.

http://codereview.appspot.com/4080047/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[anatoly techtonik]

On Tue, Feb 1, 2011 at 12:59 AM, Benjamin Peterson  wrote:

 I see no reason for b.p.o bureaucracy.
>>>
>>> It provides a place for discussion, and makes it easier to coordinate
>>> multiple efforts.
>>
>> Code review system provides a better space for discussion if we are
>> speaking about simple code cleanup. To me polluting tracker with the
>> issues that are neither bugs nor feature requests only makes bug
>> triaging process and search more cumbersome.
>
> If it's not a bug or a feature request, why does it need to change?

Because code cleanup patches pave road for more complex pieces of
work. Clean code makes patches easier to review. It saves developer's
time and as a result useful patches are integrated into codebase more
quickly.
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Georg Brandl]

Am 31.01.2011 22:58, schrieb anatoly techtonik:
> On Mon, Jan 31, 2011 at 11:09 PM, Ethan Furman  wrote:
>> techto...@gmail.com wrote:
>>>
>>> I see no reason for b.p.o bureaucracy.
>>
>> It provides a place for discussion, and makes it easier to coordinate
>> multiple efforts.
> 
> Code review system provides a better space for discussion if we are
> speaking about simple code cleanup. To me polluting tracker with the
> issues that are neither bugs nor feature requests only makes bug
> triaging process and search more cumbersome.

Note that while the domain may be "bugs.python.org" (because that is a
traditional name used by many projects), it really is an *issue tracker*,
and for us, all patches are issues that should be tracked there.

A mailing list works only if you have a small group of core developers
who can independently organize the incoming mails using local tools,
such as the read/unread marking of the email client.  For a larger
group this doesn't work (how do you assign a patch to someone using
a mailing list?).  It sure is more convenient to do patch review, but
that's why we are working on Rietveld integration in the tracker.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MSI: Remove dependency from win32com.client module (issue4080047)

[Georg Brandl]

Am 31.01.2011 23:05, schrieb anatoly techtonik:
> On Mon, Jan 31, 2011 at 10:58 PM, Georg Brandl  wrote:
>> Am 31.01.2011 21:45, schrieb techto...@gmail.com:
>>> There is no b.p.o issue as it's not a bug, but a tiny copy/paste patch
>>> to clean up the code a bit while I am trying to understand how to add
>>> Python to the PATH.
>>>
>>> I see no reason for b.p.o bureaucracy. Mercurial-style workflow [1] is
>>> more beneficial to development as it doesn't require switching from
>>> console to browser for submitting changes. This way tiny changes can be
>>> integrated/updated more rapidly.
>>
>> The tracker is not bureaucracy, it's how our development process works.
> 
> Don't you want to improve this process? Code review system is a much
> better place to review patches than mailing list or bug tracker.
> Especially patches that are not related to actual bugs.

If there are patches only on the code review system, others only on the
issue tracker, and still others on both, people will get confused quickly.
There needs to be a single canonical place to collect issues, and that needs
to be the issue tracker (since bug reports cannot go anywhere else).

I'm enthusiastic about anything that *improves* this process, but you're
proposing to *change* it.

>> I know that Mercurial uses a different process, with patches always going
>> to the mailing list and being reviewed there, but that would be way too
>> much volume for python-dev considering our number of patches.
> 
> Seems reasonable. Do you have any stats how many patches are sent
> weekly and how many of them are actually integrated?

No, but you can have a look at the weekly "Summary of Python tracker issues"
emails that are sent to this list by a script.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com