Re: Looking for direction
On 05/13/2015 06:12 PM, Dave Angel wrote: On 05/13/2015 08:45 PM, 20/20 Lab wrote: You accidentally replied to me, rather than the mailing list. Please use reply-list, or if your mailer can't handle that, do a Reply-All, and remove the parts you don't want. On 05/13/2015 05:07 PM, Dave Angel wrote: On 05/13/2015 07:24 PM, 20/20 Lab wrote: I'm a beginner to python. Reading here and there. Written a couple of short and simple programs to make life easier around the office. Welcome to Python, and to this mailing list. That being said, I'm not even sure what I need to ask for. I've never worked with external data before. I have a LARGE csv file that I need to process. 110+ columns, 72k rows. That's not very large at all. In the grand scheme, I guess not. However I'm currently doing this whole process using office. So it can be a bit daunting. I'm not familiar with the office operating system. I managed to write enough to reduce it to a few hundred rows, and the five columns I'm interested in. Now is were I have my problem: myList = [ [123, XXX, Item, Qty, Noise], [72976, YYY, Item, Qty, Noise], [123, XXX ItemTypo, Qty, Noise]] It'd probably be useful to identify names for your columns, even if it's just in a comment. Guessing from the paragraph below, I figure the first two columns are account staff The columns that I pull are Account, Staff, Item Sold, Quantity sold, and notes about the sale (notes arent particularly needed, but the higher ups would like them in the report) Basically, I need to check for rows with duplicate accounts row[0] and staff (row[1]), and if so, remove that row, and add it's Qty to the original row. And which column is that supposed to be? Shouldn't there be a number there, rather than a string? I really dont have a clue how to go about this. The number of rows change based on which run it is, so I couldnt even get away with using hundreds of compare loops. If someone could point me to some documentation on the functions I would need, or a tutorial it would be a great help. Is the order significant? Do you have to preserve the order that the accounts appear? I'll assume not. Have you studied dictionaries? Seems to me the way to handle the problem is to read in a row, create a dictionary with key of (account, staff), and data of the rest of the line. Each time you read a row, you check if the key is already in the dictionary. If not, add it. If it's already there, merge the data as you say. Then when you're done, turn the dict back into a list of lists. The order is irrelevant. No, I've not really studied dictionaries, but a few people have mentioned it. I'll have to read up on them and, more importantly, their applications. Seems that they are more versatile then I thought. Thank you. You have to realize that a tuple can be used as a key, in your case a tuple of Account and Staff. You'll have to decide how you're going to merge the ItemSold, QuantitySold, and notes. Tells you how often I actually talk in mailing lists. My apologies, and thank you again. -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for direction
On 05/13/2015 06:12 PM, Dave Angel wrote: On 05/13/2015 08:45 PM, 20/20 Lab wrote: You accidentally replied to me, rather than the mailing list. Please use reply-list, or if your mailer can't handle that, do a Reply-All, and remove the parts you don't want. ...and now that you mention it. I appear to have done that with all of my replies yesterday. My deepest apologies for that. -- https://mail.python.org/mailman/listinfo/python-list
[issue24160] Pdb sometimes crashes when trying to remove a breakpoint defined in a different debugger sessoon
Xavier de Gaye added the comment:
I can reproduce the problem on python 3.5 with test3.py as:
def foo():
foo = 7789
bar = 7788
$ python
Python 3.5.0a4+ (default:8bac00eadfda, May 6 2015, 17:40:12)
[GCC 4.9.2 20150304 (prerelease)] on linux
Type help, copyright, credits or license for more information.
import pdb, test3
pdb.run('test3.foo()')
string(1)module()
(Pdb) step
--Call--
/home/xavier/tmp/test3.py(1)foo()
- def foo():
(Pdb) break 3
Breakpoint 1 at /home/xavier/tmp/test3.py:3
(Pdb) continue
/home/xavier/tmp/test3.py(3)foo()
- bar = 7788
(Pdb) continue
pdb.run('test3.foo()')
string(1)module()
(Pdb) step
--Call--
/home/xavier/tmp/test3.py(1)foo()
- def foo():
(Pdb) break # 'break' lists no breakpoints.
(Pdb) break 2
Breakpoint 2 at /home/xavier/tmp/test3.py:2
(Pdb) break # 'break' lists two breakpoints.
Num Type Disp Enb Where
1 breakpoint keep yes at /home/xavier/tmp/test3.py:3
breakpoint already hit 1 time
2 breakpoint keep yes at /home/xavier/tmp/test3.py:2
(Pdb)
On the second debugging session, the first 'break' command lists no
breakpoints, while the second 'break' command lists two breakpoints including
the one set in the first debugging session.
The problem is that the 'breaks' attribute of the Pdb instance is inconsistent
with the 'bplist' and 'bpbynumber' class attributes of the bdb.Breakpoint
class when the second debugging session is started.
--
nosy: +xdegaye
versions: +Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24160
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24195] Add `Executor.filter` to concurrent.futures
Paul Moore added the comment: Just as a note - to test a pure Pthon patch like this, you can apply the patch to your installed Python 3.4 installation, and just run the test using that. There should be no need to build your own Python. python C:\Apps\Python34\Lib\test\test_concurrent_futures.py test_cancel (__main__.FutureTests) ... ok test_cancelled (__main__.FutureTests) ... ok test_done (__main__.FutureTests) ... ok test_done_callback_already_cancelled (__main__.FutureTests) ... ok test_done_callback_already_failed (__main__.FutureTests) ... ok test_done_callback_already_successful (__main__.FutureTests) ... ok [... etc] -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24195 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24129] Incorrect (misleading) statement in the execution model documentation
levkivskyi added the comment: Since no one proposed alternative ideas, I am submitting my proposal as a patch, with the following wording: A class definition is an executable statement that may use and define names. Free variables follow the normal rules for name resolution, while unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods -- keywords: +patch Added file: http://bugs.python.org/file39383/classdoc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24129 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24013] Improve os.scandir() and DirEntry documentation
STINNER Victor added the comment: 8. Added Availability: Unix, Windows. to scandir docs like listdir and most other os functions have. That's wrong: availability should only be explained when a function is not always available. os.listdir() and os.scandir() are always available. I made the opposite change: I removed the availability information from os.listdir() and many other os functions in the changeset 77ebd3720284. Thanks Ben, I applied your patch (except of the Availability line). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24013 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Eventlet 0.17.4 released
Eventlet is a concurrent networking library for Python that allows you to
change how you run your code, not how you write it.
https://pypi.python.org/pypi/eventlet/0.17.4
First attempt at 2015-02-23, it's 4th bug fix release in 0.17 series.
Featuring full Python3 and IPv6 support! Thanks to everyone involved. I'm
already using Python3 for all my new projects and glad to report another
important library is available.
Also, Github https://github.com/eventlet/eventlet/ is now main official
repository and Bitbucket is eventually consistent mirror.
News:
* ssl: incorrect initalization of default context; Thanks to stuart-mclaren
* green.thread: Python3.3+ fixes; Thanks to Victor Stinner
* Semaphore.acquire() accepts timeout=-1; Thanks to Victor Stinner
* wsgi: Provide python logging compatibility; Thanks to Sean Dague
* greendns: fix premature connection closing in DNS proxy; Thanks to Tim Simmons
* greenio: correct fd close; Thanks to Antonio Cuni and Victor Sergeyev
* green.ssl: HTTPS client Python 2.7.9+ compatibility
* setup: tests.{isolated,manual} polluted top-level packages
* greendns: fix dns.name import and Python3 compatibility
* Full Python3 compatibility; Thanks to Jakub Stasiak
* greendns: IPv6 support, improved handling of /etc/hosts; Thanks to Floris
Bruynooghe
* tpool: make sure we return results during killall; Thanks to David Szotten
* semaphore: Don't hog a semaphore if someone else is waiting for it; Thanks to
Shaun Stanworth
* green.socket: create_connection() was wrapping all exceptions in
socket.error; Thanks to Donagh McCabe
* Make sure SSL retries are done using the exact same data buffer; Thanks to
Lior Neudorfer
* greenio: shutdown already closed sockets without error; Thanks to David
Szotten
Our website: http://eventlet.net/
Direct package download links:
https://pypi.python.org/packages/3.4/e/eventlet/eventlet-0.17.4-py2.py3-none-any.whl
https://pypi.python.org/packages/source/e/eventlet/eventlet-0.17.4.tar.gz
--
https://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
[issue24187] del statement documentation doesn't mention name binding behaviour
Jon added the comment: Sorry, I think I just misread this section. I was confused by the fact that del binds names like assignment does, so that the following tries to delete a local name and fails: x = 1 def f(): del x f() In fact the documentation does say that there must be global statement in the block for del to delete a global name, so my bad. -- resolution: - not a bug status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
Gregory Ewing greg.ew...@canterbury.ac.nz: BartC wrote: It appears to be those = and + operations in the code above where much of the time is spent. When I trace out the execution paths a bit more, I'll have a better picture of how many lines of C code are involved in each iteration. The path from decoding a bytecode to the C code that implements it can be rather convoluted, but there are reasons for each of the complications -- mainly to do with supporting the ability to override operators with C and/or Python code. If you removed those abilities, the implemention would be simpler, and possibly faster. But then the language wouldn't be Python any more. I agree that Python's raison-d'être is its dynamism and expressive power. It definitely shouldn't be sacrificed for performance. However, in some respects, Python might be going overboard with its dynamism; are all those dunder methods really needed? Must false be defined so broadly? Must a method lookup necessarily involve object creation? Marko -- https://mail.python.org/mailman/listinfo/python-list
[issue24130] Remove -fno-common compile option from OS X framework builds?
Ronald Oussoren added the comment: According to [1] common symbols are not allowed in frameworks. I guess that's why we added '-fno-common' to the linker flags. [1] https://developer.apple.com/library/mac/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkBinding.html#//apple_ref/doc/uid/20002256-BAJICBDD -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24130 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22547] Implement an informative `BoundArguments.__repr__`
Serhiy Storchaka added the comment:
May be omit names for positionalarguments?
def foo(a, *args, b=10, **kwargs): pass
...
inspect.signature(foo).bind(1, 2, 3, b=4, c=5)
BoundArguments at 0xb6eee9ec (a=1, args=(2, 3), b=4, kwargs={'c': 5})
I think it would look better as:
BoundArguments (1, 2, 3, b=4, c=5)
--
nosy: +serhiy.storchaka
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22547
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24195] Add `Executor.filter` to concurrent.futures
Ram Rachum added the comment: Patch with documentation attached. (I don't know how to concatenate patches, so 2.patch contains only the documentation, while 1.patch has the implementation and the tests (but Ethan's patch is better.)) Brian, regarding your simpler implementation based on `Executor.map`: It looks good to me, but I'm not sure if it passes the tests. If it does then I don't mind if that's the implementation that would be used. -- Added file: http://bugs.python.org/file39381/2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24195 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24195] Add `Executor.filter` to concurrent.futures
Antoine Pitrou added the comment: This feature looks unnecessary to me as well. Adding features has a non-zero cost in maintenance. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24195 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa ma...@pacujo.net wrote: However, in some respects, Python might be going overboard with its dynamism; are all those dunder methods really needed? Yes - at least, most of them. As regards operators, there are three options: either you have magic methods for all of them (Python style), or none of them (Java style, no operator overloading), or you hybridize and permit just a handful of them (and then you have to decide which). There's really no reason not to have them. The other dunder methods are a mixed bag; some are to allow you to customize object creation itself (a class's __new__ method could be considered equivalent to an instance-specific __call__ method on the type object), some let you pretend to be different types of number (__int__, __index__, __complex__), which allows you to duck-type integerness rather than having to subclass int and rely on magic; and others let you customize the behaviour of well-known functions, such as __len__ for len() and __repr__ for repr(). Without dunder methods for all of these, it would be difficult to make an object play nicely with the overall Python ecosystem. There are others, though, which are less crucial (__getstate__ and so on for pickle), but you can ignore them if you're not using those features. Must false be defined so broadly? Different languages define true and false differently. REXX says that 1 is true and 0 is false, and anything else is an error. Pike says that the integer 0 is false and anything else is true; the philosophy is that a thing is true and the absence of any thing is false. Python says that an empty thing is false and a non-empty thing is true; if next(iter(x)) raises StopIteration, x is probably false, and vice versa. All three have their merits, all three have their consequences. One consequence of Python's model is that a __bool__ method is needed on any object that might be empty (unless it defines __len__, in which case that makes a fine fall-back); it's normal in Python code to distinguish between if x: and if x is not None:, where the former sees if x has anything in it, but the latter sees if there x even exists. (More or less.) Must a method lookup necessarily involve object creation? Actually, no. Conceptually, this method call: foo.bar(1, 2, 3) involves looking up 'foo' in the current namespace, looking up attribute 'bar' on it, treating the result as a function, and calling it with three integer objects as its arguments. And the language definition demands that this work even if the foo.bar part is broken out: def return_function(): return foo.bar def call_function(): return_function()(1, 2, 3) But a particular Python implementation is most welcome to notice the extremely common situation of method calls and optimize it. I'm not sure if PyPy does this, but I do remember reading about at least one Python that does; CPython has an optimization for the actual memory allocations involved, though I think it does actually construct some sort of object for each one; as long as the resulting behaviour is within spec, objects needn't be created just to be destroyed. Dynamism doesn't have to be implemented naively, just as long as the slow path is there if anyone needs it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
[issue18378] locale.getdefaultlocale() fails on Mac OS X with default language set to English
Ronald Oussoren added the comment: Dimitry's patch looks good, I added my patch before checking if there already is patch. The only thing that might be cause discussion is when to accept 'UTF-8' as a valid locale name. My patch only accepts in on OSX, while Dimitry's patch accepts it everywwhere. Writing this I'm slightly in favour of Dimitry's approach: I quite often run into problems when using SSH to log in to a Linux box from my OSX laptop (with LC_CTYPE=UTF-8). Almost everything works correctly, except for Python code that uses the locale module (which craps out with the exception in the first message in this issue). IMHO Dimitry's patch should be applied as is. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18378 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24200] Redundant id in informative reprs
New submission from Serhiy Storchaka: import inspect def foo(a, *, b=10): pass ... inspect.signature(foo) Signature at 0xb6e2768c (a, *, b=10) I think the id is not needed in informative repr if you implemented __eq__. Identity doesn't matter if different instances can be equal. The id in the repr only adds a noise. -- components: Library (Lib) messages: 243247 nosy: serhiy.storchaka, yselivanov priority: normal severity: normal status: open title: Redundant id in informative reprs type: behavior versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24200 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24201] _winreg PyHKEY Type Confusion
New submission from JohnLeitch:
The Python _winreg module suffers from a type confusion vulnerability wherein
pointers can be passed directly in place of PyHKEY instances e.g.
_winreg.QueryValue(0x41414141, )
This behavior is due to the underlying PyHKEY_AsHKEY function of _winreg.c:
BOOL
PyHKEY_AsHKEY(PyObject *ob, HKEY *pHANDLE, BOOL bNoneOK)
{
if (ob == Py_None) {
if (!bNoneOK) {
PyErr_SetString(
PyExc_TypeError,
None is not a valid HKEY in this context);
return FALSE;
}
*pHANDLE = (HKEY)0;
}
else if (PyHKEY_Check(ob)) {
PyHKEYObject *pH = (PyHKEYObject *)ob;
*pHANDLE = pH-hkey;
}
else if (PyInt_Check(ob) || PyLong_Check(ob)) { if ob is an
int/long, this path is taken.
/* We also support integers */
PyErr_Clear();
*pHANDLE = (HKEY)PyLong_AsVoidPtr(ob); ob is casted to a void*
here
if (PyErr_Occurred())
return FALSE;
}
else {
PyErr_SetString(
PyExc_TypeError,
The object is not a PyHKEY object);
return FALSE;
}
return TRUE;
}
When *ob is an integer or long, the function casts it to a void*. This behavior
can be triggered using many of the _winreg functions, such as QueryValue,
QueryValueEx, EnumValue, etc.
0:000 r
eax=41414140 ebx=0027fbc8 ecx= edx= esi=770e351e edi=
eip=74bf9af3 esp=0027f738 ebp=0027f764 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
RPCRT4!NDRCContextBinding+0x13:
74bf9af3 81780498badcfe cmp dword ptr [eax+4],0FEDCBA98h
ds:002b:41414144=
0:000 k
ChildEBP RetAddr
0027f764 74c0390c RPCRT4!NDRCContextBinding+0x13
0027f774 74c86dce RPCRT4!ExplicitBindHandleMgr+0x33
0027fba8 770e625c RPCRT4!NdrClientCall2+0x2ea
0027fbc0 771041e2 ADVAPI32!SafeBaseRegQueryInfoKey+0x24
0027fc04 76eacdca ADVAPI32!RemoteRegQueryInfoKeyWrapper+0x42
0027fcbc 1e0de85c KERNELBASE!LocalOpenPerformanceText+0x1c60
0027fd14 1e0ac6fc python27!PyEnumValue+0x6c [c:\build27\cpython\pc\_winreg.c @
1213]
0027fd58 1e0efabf python27!_PyObject_GenericGetAttrWithDict+0x12c
[c:\build27\cpython\objects\object.c @ 1428]
0027fde8 1e0f27eb python27!PyEval_EvalFrameEx+0x1cdf
[c:\build27\cpython\python\ceval.c @ 2269]
0027fe00 1e0f11b2 python27!compiler_free+0x3b
[c:\build27\cpython\python\compile.c @ 322]
0027fe2c 1e11707a python27!PyEval_EvalCode+0x22
[c:\build27\cpython\python\ceval.c @ 672]
0027fe44 1e1181c5 python27!run_mod+0x2a [c:\build27\cpython\python\pythonrun.c
@ 1371]
0027fe64 1e118760 python27!PyRun_FileExFlags+0x75
[c:\build27\cpython\python\pythonrun.c @ 1358]
0027fea4 1e1190d9 python27!PyRun_SimpleFileExFlags+0x190
[c:\build27\cpython\python\pythonrun.c @ 950]
0027fec0 1e038d35 python27!PyRun_AnyFileExFlags+0x59
[c:\build27\cpython\python\pythonrun.c @ 753]
0027ff3c 1d00116d python27!Py_Main+0x965 [c:\build27\cpython\modules\main.c @
643]
0027ff80 74d57c04 python!__tmainCRTStartup+0x10f
[f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 586]
0027ff94 7741ad1f KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 7741acea ntdll!__RtlUserThreadStart+0x2f
0027ffec ntdll!_RtlUserThreadStart+0x1b
0:000 !analyze -v
***
* *
*Exception Analysis *
* *
***
FAULTING_IP:
RPCRT4!NDRCContextBinding+13
74bf9af3 81780498badcfe cmp dword ptr [eax+4],0FEDCBA98h
EXCEPTION_RECORD: -- (.exr 0x)
ExceptionAddress: 74bf9af3 (RPCRT4!NDRCContextBinding+0x0013)
ExceptionCode: c005 (Access violation)
ExceptionFlags:
NumberParameters: 2
Parameter[0]:
Parameter[1]: 41414144
Attempt to read from address 41414144
CONTEXT: -- (.cxr 0x0;r)
eax=41414140 ebx=0027fbc8 ecx= edx= esi=770e351e edi=
eip=74bf9af3 esp=0027f738 ebp=0027f764 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
RPCRT4!NDRCContextBinding+0x13:
74bf9af3 81780498badcfe cmp dword ptr [eax+4],0FEDCBA98h
ds:002b:41414144=
FAULTING_THREAD: 0274
DEFAULT_BUCKET_ID: INVALID_POINTER_READ
PROCESS_NAME: python.exe
ERROR_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%08lx referenced
memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%08lx referenced
memory at 0x%08lx. The memory could not be %s.
EXCEPTION_PARAMETER1:
EXCEPTION_PARAMETER2: 41414144
READ_ADDRESS: 41414144
FOLLOWUP_IP:
Re: Building CPython
On 15/05/2015 07:05, Gregory Ewing wrote: BartC wrote: It appears to be those = and + operations in the code above where much of the time is spent. When I trace out the execution paths a bit more, I'll have a better picture of how many lines of C code are involved in each iteration. The path from decoding a bytecode to the C code that implements it can be rather convoluted, but there are reasons for each of the complications -- mainly to do with supporting the ability to override operators with C and/or Python code. If you removed those abilities, the implemention would be simpler, and possibly faster. But then the language wouldn't be Python any more. That's the challenge; programs must still work as they did before. (But I suppose it can be exasperating for CPython developers if 99% of programs could be made faster but can't be because of the 1% that rely on certain features). Still, I'm just seeing what there is in CPython that limits the performance, and whether anything can be done /easily/ without going to solutions such as PyPy which are unsatisfactory IMO (by being even more fantastically complicated, but they don't always give a speed-up either). For example, there is a /specific/ byte-code called BINARY_ADD, which then proceeds to call a /generic/ binary-op handler! This throws away the advantage of knowing at byte-code generation time exactly which operation is needed. (Unless I'm just looking at a bunch of macros or maybe there is some inlining going on with the compiler reducing all those table-indexing operations for 'Add', with direct accesses, but it didn't look like it. I'm just glad it doesn't use C++ which would have made it truly impossible to figure out what's going on.) (BTW since I'm having to use Linux to compile this anyway, is there a tool available that will tell me whether something in the C sources is a function or macro? (And preferably where the definition might be located.)) -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
BartC wrote: It appears to be those = and + operations in the code above where much of the time is spent. When I trace out the execution paths a bit more, I'll have a better picture of how many lines of C code are involved in each iteration. The path from decoding a bytecode to the C code that implements it can be rather convoluted, but there are reasons for each of the complications -- mainly to do with supporting the ability to override operators with C and/or Python code. If you removed those abilities, the implemention would be simpler, and possibly faster. But then the language wouldn't be Python any more. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
[issue22547] Implement an informative `BoundArguments.__repr__`
Serhiy Storchaka added the comment: Here is an implementation. -- keywords: +patch Added file: http://bugs.python.org/file39380/BoundArguments_repr_alt.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22547 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Looking for direction
maybe we can change this list to dict, using item[0] and item[1] as keys,
the whole item as value . then you can update by the same key i think
Tim Chase python.l...@tim.thechases.com于2015年5月15日 周五01:17写道：
On 2015-05-14 09:57, 20/20 Lab wrote:
On 05/13/2015 06:23 PM, Steven D'Aprano wrote:
I have a LARGE csv file that I need to process. 110+ columns,
72k rows. I managed to write enough to reduce it to a few
hundred rows, and the five columns I'm interested in.
I actually stumbled across the csv module after coding enough to
make a list of lists. So that is more the reason I approached the
list; Nothing like spending hours (or days) coding something that
already exists and just dont know about.
Now is were I have my problem:
myList = [ [123, XXX, Item, Qty, Noise],
[72976, YYY, Item, Qty, Noise],
[123, XXX ItemTypo, Qty, Noise]]
Basically, I need to check for rows with duplicate accounts
row[0] and staff (row[1]), and if so, remove that row, and add
it's Qty to the original row. I really dont have a clue how to
go about this.
processed = {} # hold the processed data in a dict
for row in myList:
account, staff = row[0:2]
key = (account, staff) # Put them in a tuple.
if key in processed:
# We've already seen this combination.
processed[key][3] += row[3] # Add the quantities.
else:
# Never seen this combination before.
processed[key] = row
newlist = list(processed.values())
It does, immensely. I'll make this work. Thank you again for the
link from yesterday and apologies for hitting the wrong reply
button. I'll have to study more on the usage and implementations
of dictionaries and tuples.
In processing the initial CSV file, I suspect that using a
csv.DictReader would make the code a bit cleaner. Additionally,
as you're processing through the initial file, unless you need
the intermediate data, you should be able to do it in one pass.
Something like
HEADER_ACCOUNT = account
HEADER_STAFF = staff
HEADER_QTY = Qty
processed = {}
with open(data.csv) as f:
reader = csv.DictReader(f)
for row in reader:
if should_process_row(row):
account = row[HEADER_ACCOUNT]
staff = row[HEADER_STAFF]
qty = row[HEADER_QTY]
try:
row[HEADER_QTY] = qty = int(qty)
except Exception:
# not a numeric quantity?
continue
# from Steven's code
key = (account, staff)
if key in processed:
processed[key][HEADER_QTY] += qty
else:
processed[key][HEADER_QTY] = row
so_something_with(processed.values())
I find that using names is a lot clearer than using arbitrary
indexing. Barring that, using indexes-as-constants still would
add further clarity.
-tkc
.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
wxjmfa...@gmail.com: Implement unicode correctly. Did they reject your patch? Marko -- https://mail.python.org/mailman/listinfo/python-list
[issue23085] update internal libffi copy to 3.2.1
Marc-Andre Lemburg added the comment: issue23042 is fixed now. libffi 3.2.1 apparently has the same issue, so the issue23042 patch would probably have to be reapplied (slightly modified, though). Seeing that libffi has had a major compilation problem breaking it on at least FreeBSD and most probably a lot of other x86 based systems as well, I don't really have much confidence in the libffi maintenance, so the usual latest is the greatest may not be the best approach. Are there any issue *we* have with libffi which an update to 3.2.1 would solve ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23085 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18378] locale.getdefaultlocale() fails on Mac OS X with default language set to English
Changes by Ronald Oussoren ronaldousso...@mac.com: Added file: http://bugs.python.org/file39384/issue-18378-py27.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18378 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18378] locale.getdefaultlocale() fails on Mac OS X with default language set to English
Changes by Ronald Oussoren ronaldousso...@mac.com: Added file: http://bugs.python.org/file39385/issue-18378-py35.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18378 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Python: How to find out values of all feasible x under constraints.
Xiang Zhang wrote: I want to know how to find out values of all feasible x under constraints. x = [x_1, x_2, x_3,..., x_10] constraints: x_i = 0,1,2,3 or 4, where i=1,2,10 x_1 + x_2 + x_3 +...+x_10 = 15 That are 5**10 == 9765625 candidates. That's still feasible to check using brute force. How to find out all the feasible solutions x (domain of x) using python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the code? Any hint or help would be highly appreciated! solutions = [x for x in itertools.product(range(5), repeat=10) if sum(x) = 15] len(solutions) 1556215 The first ten solutions: solutions[:10] [(0, 0, 0, 0, 0, 0, 0, 0, 0, 0), (0, 0, 0, 0, 0, 0, 0, 0, 0, 1), (0, 0, 0, 0, 0, 0, 0, 0, 0, 2), (0, 0, 0, 0, 0, 0, 0, 0, 0, 3), (0, 0, 0, 0, 0, 0, 0, 0, 0, 4), (0, 0, 0, 0, 0, 0, 0, 0, 1, 0), (0, 0, 0, 0, 0, 0, 0, 0, 1, 1), (0, 0, 0, 0, 0, 0, 0, 0, 1, 2), (0, 0, 0, 0, 0, 0, 0, 0, 1, 3), (0, 0, 0, 0, 0, 0, 0, 0, 1, 4)] -- https://mail.python.org/mailman/listinfo/python-list
Re: Feature Request: Reposition Execution
Am 15.05.15 um 05:58 schrieb Skybuck Flying: Thanks for the ideas, I haven't tried them yet. I wonder if they will work in a multi-threaded fashion. I doubt it. The run_script runs in it's own thread. It would be of enormous help if you would create a minimal script just like the above for your situation. What does it mean, runs in a thread? How do you achieve that from Jython? The exception would have to be raise from another thread. (The other thread check for errors and needs to abort the run script). Well usually you can pass messages between the threads, wait for termination etc. I've got no experience with threading in Jython; however this site http://www.jython.org/jythonbook/en/1.0/Concurrency.html suggests that there is a large number of primitives available to do that. Concerning Resume, what do you expect: A) starting afresh from the beginning or B) continue the execution at the point where the exception was thrown? Because A) is/should be easy, B) could be impossible Christian -- https://mail.python.org/mailman/listinfo/python-list
[issue15582] Enhance inspect.getdoc to follow inheritance chains
Martin Panter added the comment: getdoc-news.patch suggests some wording to add to What’s New, and also adds a “Changed in version 3.5” note to inspect.getdoc(). BTW I also noticed that the class doc strings are not inherited from object.__doc__, although method doc strings _are_ inherited from object(), such as object.__init__.__doc__. The current documentation suggests that the class doc string “The most base type” should also be inherited. $ cat module.py class SomeClass: '''CLASS DOCSTRING''' def __init__(self): '''METHOD DOCSTRING''' $ ./python -m pydoc module.SomeClass # Doc strings intact [. . .] module.SomeClass = class SomeClass(builtins.object) | CLASS DOCSTRING | | Methods defined here: | | __init__(self) | METHOD DOCSTRING | [. . .] $ ./python -OOm pydoc module.SomeClass # Method inherited, class stripped [. . .] module.SomeClass = class SomeClass(builtins.object) | Methods defined here: | | __init__(self) | Initialize self. See help(type(self)) for accurate signature. | [. . .] I also wonder how well this feature would work when someone tries to override a base method by using a mix-in type class. -- Added file: http://bugs.python.org/file39379/getdoc-news.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15582 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
new python.org mailing list, elections-wg
It is for discussing voting software (currently Helios and Evote) with the end result that the new PSF election Commissioner Ian Cordasco will pick one and use it for the next PSF eÃlection. We're hoping to turn it into a real PSF workgroup. It looks like it is going to be a fairly nerdy place. All are welcome, this is a public list. Interested parties please join at: https://mail.python.org/mailman/listinfo/elections-wg Thank you. Laura -- https://mail.python.org/mailman/listinfo/python-list
[issue24013] Improve os.scandir() and DirEntry documentation
Roundup Robot added the comment: New changeset e7c7431f91b2 by Victor Stinner in branch 'default': Closes #24013: Improve os.scandir() and DirEntry documentation https://hg.python.org/cpython/rev/e7c7431f91b2 -- nosy: +python-dev resolution: - fixed stage: - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24013 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
wxjmfa...@gmail.com: Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit : wxjmfa...@gmail.com: Implement unicode correctly. Did they reject your patch? You can not patch something that is wrong by design. Are you saying the Python language spec is unfixable or that the CPython implementation is unfixable? If CPython is unfixable, you can develop a better Python implementation. If Python itself is unfixable, what brings you here? Marko -- https://mail.python.org/mailman/listinfo/python-list
[issue24199] Idle: remove idlelib.idlever.py and its use in About dialog
Serhiy Storchaka added the comment: Are there third-party IDLE plugins? If yes, this change can break them. -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24199 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Python: How to find out values of all feasible x under constraints.
On 15/05/2015 04:58, Xiang Zhang wrote: Dear all, I am writing a code using Python now. I want to know how to find out values of all feasible x under constraints. x = [x_1, x_2, x_3,..., x_10] constraints: x_i = 0,1,2,3 or 4, where i=1,2,10 x_1 + x_2 + x_3 +...+x_10 = 15 How to find out all the feasible solutions x (domain of x) using python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the code? Any hint or help would be highly appreciated! Sincerely, Xiang Zhang There are several constraint libraries on pypi if you don't want to roll your own. See for example https://pypi.python.org/pypi/python-constraint/1.2 -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, 15 May 2015 08:52 pm, Marko Rauhamaa wrote: wxjmfa...@gmail.com: Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit : wxjmfa...@gmail.com: Implement unicode correctly. Did they reject your patch? You can not patch something that is wrong by design. Are you saying the Python language spec is unfixable or that the CPython implementation is unfixable? JMF is obsessed with a trivial and artificial performance regression in the handling of Unicode strings since Python 3.3, which introduced a significant memory optimization for Unicode strings. Each individual string uses a code unit no larger than necessary, thus if a string contains nothing but ASCII or Latin 1 characters, it will use one byte per character; if it fits into the Basic Multilingual Plane, two bytes per character; and only use four bytes per character if there are astral characters in the string. (That is, Python strings select from a Latin-1, UCS-2 and UTF-32 encoded form at creation time, according to the largest code point in the string.) The benefit of this is that most strings will use 1/2 or 1/4 of the memory that they otherwise would need, which gives an impressive memory saving. That leads to demonstrable speed-ups in real-world code, however it is possible to find artificial benchmarks that experience a slowdown compared to Python 3.2. JMF found one such artificial benchmark, involving creating and throwing away many strings as fast as possible without doing any work with them, and from this has built this fantasy in his head that Python is not compliant with the Unicode spec and is logically, mathematically broken. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, May 15, 2015 at 8:14 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: (If anything, using an implicit boolean test will be faster than an explicit manual test, because it doesn't have to call the len() global.) Even more so: Some objects may be capable of determining their own lengths, but can ascertain their own emptiness more quickly. So len(x) might have to chug chug chug to figure out exactly how many results there are (imagine a database query or something), where bool(x) merely has to see whether or not a single one exists (imagine a database query with LIMIT 1 tacked on). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
Chris Angelico ros...@gmail.com: On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa ma...@pacujo.net wrote: Must a method lookup necessarily involve object creation? Actually, no. [...] a particular Python implementation is most welcome to notice the extremely common situation of method calls and optimize it. I'm not sure that is feasible given the way it has been specified. You'd have to prove the class attribute lookup produces the same outcome in consecutive method references. Also: class X: ... def f(self): pass ... x = X() f = x.f ff = x.f f is ff False Would a compliant Python implementation be allowed to respond True? Maybe. At least method objects seem immutable: f.__name__ 'f' f.__name__ = 'g' Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'method' object has no attribute '__name__' f.__eq__ = None Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'method' object attribute '__eq__' is read-only f.xyz = 'xyz' Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'method' object has no attribute 'xyz' Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
Am 14.05.15 um 20:50 schrieb Terry Reedy: On 5/14/2015 1:11 PM, Chris Angelico wrote: 2) make test - run the entire test suite. Takes just as long every time, but most of it won't have changed. The test runner has an option, -jn, to run tests in n processes instead of just 1. On my 6 core pentium, -j5 cuts time to almost exactly 1/5th of otherwise. -j10 seems faster but have not times it. I suspect that 'make test' does not use -j option. Just to clarify, -j is an option of GNU make to run the Makefile in parallel. Unless the Makefile is buggy, this should result in the same output. You can also set an environment variable to enable this permanently (until you log out) like export MAKEFLAGS=-j5 Put this into your .bashrc or .profile, and it'll become permanent. Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: Trying to build Python
On Fri, May 15, 2015 at 9:02 PM, Rustom Mody rustompm...@gmail.com wrote: On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote: I am trying to build Python 3 on a Debian system. It is successful, but a few things where missing. Partly I solved it, but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl _gdbm What do I need to do to get those resolved also? $ aptitude build-dep python3 should fetch the requirements of python3 And if not, try grabbing some development libraries: $ sudo apt-get install libbz2-dev libsqlite3-dev libreadline-dev libssl-dev libgdbm-dev I'm not sure about the _dbm module, not sure what it needs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, May 15, 2015 at 10:10 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: The benefit of this is that most strings will use 1/2 or 1/4 of the memory that they otherwise would need, which gives an impressive memory saving. That leads to demonstrable speed-ups in real-world code, however it is possible to find artificial benchmarks that experience a slowdown compared to Python 3.2. It's also possible to find a number of situations in which a narrow build of 3.2 was faster than 3.3, due to the buggy handling of surrogates. I've no idea whether jmf is still complaining against that basis or not, as I don't see his posts. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Trying to build Python
I am trying to build Python 3 on a Debian system. It is successful, but a few things where missing. Partly I solved it, but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl _gdbm What do I need to do to get those resolved also? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, 15 May 2015 06:59 pm, Marko Rauhamaa wrote: However, in some respects, Python might be going overboard with its dynamism; are all those dunder methods really needed? Must false be defined so broadly? Must a method lookup necessarily involve object creation? Yes, what do you mean, and no. (1) Yes, the dunder methods are necessary to support operator overloading and various other protocols. The existence of dunder methods doesn't have any runtime costs except for that related to memory usage. Fortunately, the dunder methods only exist on the class itself, not each and every instance. (2) What do you mean by false being defined so broadly? The singleton instance False is not defined broadly at all. It is a built-in constant, and there's only one of it. In Python 3, False is even a keyword, so you cannot redefine the name. (It's not a keyword in Python 2 because of historical reasons.) Perhaps you are talking about falsey (false-like) instances, e.g. if []: ... else: ... will run the else block. That's quite broad, in a sense, but it has no real runtime cost over and above a more Pascal-ish language would force you to have: if len([]) == 0: ... else: ... Because True and False are singletons, the overhead of testing something in a boolean context is no greater than the cost of testing the same condition by hand. That is, it makes no difference whether you manually compare the list's length to zero, or let its __nonzero__ or __bool__ method do the same. (If anything, using an implicit boolean test will be faster than an explicit manual test, because it doesn't have to call the len() global.) (3) Method lookups don't *necessarily* have to involve object creation. The only semantics which Python requires (so far as I understand it) are: - Taking a reference to an unbound method: ref = str.upper may return the function object itself, which clearly already exists. - Taking a reference to a bound method: ref = some string.upper must return a method object, but it doesn't have to be recreated from scratch each and every time. It could cache it once, then always return that. That, I believe, is an implementation detail. - Calling a method may bypass creating a method, and just use the function object directly, provided Python knows that the descriptor has no side-effects. For example, since FunctionType.__get__ has no side-effects, a Python interpreter could special-case function objects and avoid calling the descriptor protocol when it knows that the method is just going to be called immediately. But... having said all that... how do you know that these issues are bottlenecks that eliminating them would speed up the code by any significant amount? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Trying to build Python
On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote: I am trying to build Python 3 on a Debian system. It is successful, but a few things where missing. Partly I solved it, but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl _gdbm What do I need to do to get those resolved also? $ aptitude build-dep python3 should fetch the requirements of python3 -- https://mail.python.org/mailman/listinfo/python-list
[issue24202] Multiprocessing Pool not working for userdefined function
Abhishek Manandhar added the comment: Yes actually it produce no erroe on standalone script. But the script executes without any outputs. not even for numpy function. I used code below in script. import multiprocessing import numpy def f(x): return x*x if __name__ = __main__: p= multiprocessing.Pool(5) print p.map(numpy.sqrt,[1,2,3,4]) print p.map(f,[1,2,3,4]) On May 15, 2015 2:52 PM, Paul Moore rep...@bugs.python.org wrote: Paul Moore added the comment: Multiprocessing works by firing up additional processes. Those processes won't have access to functions defined in the interactive interpreter. Can you reproduce this problem in a standalone script? I suspect not, but if you can please post the script here. Marking as not a bug, but I haven't closed it yet in case a script reproducing the issue can be provided. -- nosy: +paul.moore resolution: - not a bug ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24202 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24202 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
pip grabs tar.gz file instead of whl?
I was fed up with trying to install from pypi to Windows. Setup.py more often than not wouldn't be able to find the VS compiler. So I thought I'd try the direct route to the excellent Christoph Gohlke site at http://www.lfd.uci.edu/~gohlke/pythonlibs/ which is all whl files these days. However as you can see below despite my best efforts I'm still processing the tar.gz file, so what am I doing wrong? C:\Users\Mark\Documents\MyPythonpip install --no-cache-dir --trusted-host http://www.lfd.uci.edu/ -U -f http://www.lfd.uci.edu/~gohlke/pythonlibs/ numba Collecting numba This repository located at www.lfd.uci.edu is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host www.lfd.uci.edu'. DEPRECATION: Implicitly allowing locations which are not hosted at a secure origin is deprecated and will require the use of --trusted-host in the future. Downloading numba-0.18.2.tar.gz (507kB) 100% || 507kB 373kB/s I'm aware of the work around, download the file, then install the local copy, but that's a whole extra step :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
[issue24033] Update _test_multiprocessing.py to use script helpers
Changes by Davin Potts pyt...@discontinuity.net: -- stage: - patch review type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24033 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23699] Add a macro to ease writing rich comparisons
Petr Viktorin added the comment: What can I, not a core developer, do to resolve this disagreement? Should I submit a PEP? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23699 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24202] Multiprocessing Pool not working for userdefined function
Paul Moore added the comment: Multiprocessing works by firing up additional processes. Those processes won't have access to functions defined in the interactive interpreter. Can you reproduce this problem in a standalone script? I suspect not, but if you can please post the script here. Marking as not a bug, but I haven't closed it yet in case a script reproducing the issue can be provided. -- nosy: +paul.moore resolution: - not a bug ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24202 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24198] please align the platform tag for windows
Paul Moore added the comment: So just to be clear - this proposal would *only* affect the tagged filename used for loading .pyd files? (And in practice, the untagged form is normally used for Windows .pyd files anyway...) -- nosy: +paul.moore ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24198 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On 15/05/2015 11:52, Marko Rauhamaa wrote: wxjmfa...@gmail.com: Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit : wxjmfa...@gmail.com: Implement unicode correctly. Did they reject your patch? You can not patch something that is wrong by design. Are you saying the Python language spec is unfixable or that the CPython implementation is unfixable? If CPython is unfixable, you can develop a better Python implementation. If Python itself is unfixable, what brings you here? Marko I forgot to mention earlier that I report all his rubbish as abuse on google groups. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, 15 May 2015 08:50 pm, Marko Rauhamaa wrote: Chris Angelico ros...@gmail.com: On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa ma...@pacujo.net wrote: Must a method lookup necessarily involve object creation? Actually, no. [...] a particular Python implementation is most welcome to notice the extremely common situation of method calls and optimize it. I'm not sure that is feasible given the way it has been specified. You'd have to prove the class attribute lookup produces the same outcome in consecutive method references. Sure. But some implementations may have a more, um, flexible approach to correctness, and offer more aggressive optimizations which break the letter of Python's semantics but work for 90% of cases. Just because CPython doesn't do so, doesn't mean that some new implementation might not offer a series of aggressive optimizations which the caller (or maybe the module?) can turn on as needed, e.g.: - assume methods never change; - assume classes are static; - assume built-in names always refer to the known built-in; etc. Such an optimized Python, when running with those optimizations turned on, is not *strictly* Python, but buyer beware applies here. If the optimizations break your code or make testing hard, don't use it. Also: class X: ... def f(self): pass ... x = X() f = x.f ff = x.f f is ff False Would a compliant Python implementation be allowed to respond True? Certainly. When you retrieve x.f, Python applies the usual attribute lookup code, which simplified looks like this: if 'f' in x.__dict__: attr = x.__dict__['f'] else: for K in type(x).__mro__: # Walk the parent classes of x in the method resolution order if 'f' in K.__dict__: attr = K.__dict__['f'] break else: # no break raise AttributeError # if we get here, we know x.f exists and is bound to attr # now apply the descriptor protocol (simplified) if hasattr(attr, '__get__'): attr = attr.__get__(x, type(x)) # Finally we can call x.f() return attr(x, *args) Functions have a __get__ method which returns the method object! Imagine they look something like this: class FunctionType: def __call__(self, *args, **kwargs): # Actually call the code that does stuff def __get__(self, instance, cls): if cls is None: # Unbound instance return self return MethodType(self, instance) # self is the function This implementation creates a new method object every time you look it up. But functions *could* do this: def __get__(self, instance, cls): if cls is None: # Unbound instance return self if self._method is None: self._method = MethodType(self, instance) # Cache it. return self._method What's more, a compliant implementation could reach the if we get here point in the lookup procedure above, and do this: # if we get here, we know attr exists if type(attr) is FunctionType: # Fast pointer comparison. return attr(x, *args) else: # do the descriptor protocol thing, and then call attr It can only do this if it knows that x.f is a real function, not some sort of callable or function subclass, because in that case who knows what side-effects the __get__ method might have. How much time would it save? Probably very little. After all, unless the method call itself did bugger-all work, the time to create the method object is probably insignificant. But it's a possible optimization. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
[issue24202] Multiprocessing Pool not working for userdefined function
New submission from Abhishek Manandhar: I was looking to implement multiprocess pool. It worked fine with the numpy function while with the user defined function it ran into error. import numpy import multiprocessing P = multiprocessing.Pool(5) P.map(numpy.sqrt,range(50)) [0.0, 1.0, 1.4142135623730951, 1.7320508075688772, 2.0, 2.2360679774997898, 2.4494897427831779, 2.6457513110645907, 2.8284271247461903, 3.0, 3.1622776601683795, 3.3166247903553998, 3.4641016151377544, 3.6055512754639891, 3.7416573867739413, 3.872983346207417, 4.0, 4.1231056256176606, 4.2426406871192848, 4.358898943540674, 4.4721359549995796, 4.5825756949558398, 4.6904157598234297, 4.7958315233127191, 4.8989794855663558, 5.0, 5.0990195135927845, 5.196152422706632, 5.2915026221291814, 5.3851648071345037, 5.4772255750516612, 5.5677643628300215, 5.6568542494923806, 5.7445626465380286, 5.8309518948453007, 5.9160797830996161, 6.0, 6.0827625302982193, 6.164414002968976, 6.2449979983983983, 6.324555320336759, 6.4031242374328485, 6.4807406984078604, 6.5574385243020004, 6.6332495807107996, 6.7082039324993694, 6.7823299831252681, 6.8556546004010439, 6.9282032302755088, 7.0] def f(x): return x*x P.map(f, range(50)) Exception in thread Thread-2: Traceback (most recent call last): File C:\Python27\lib\threading.py, line 530, in __bootstrap_inner self.run() File C:\Python27\lib\threading.py, line 483, in run self.__target(*self.__args, **self.__kwargs) File C:\Python27\lib\multiprocessing\pool.py, line 285, in _handle_tasks put(task) TypeError: expected string or Unicode object, NoneType found -- components: Windows messages: 243266 nosy: abheeman, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Multiprocessing Pool not working for userdefined function type: crash versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24202 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On 15/05/2015 10:20, Marko Rauhamaa wrote: wxjmfa...@gmail.com: Implement unicode correctly. Did they reject your patch? Marko Please don't feed him, it's been obvious for years that he hasn't the faintest idea what he's talking about. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Sat, May 16, 2015 at 1:00 AM, Ian Kelly ian.g.ke...@gmail.com wrote: On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: How much time would it save? Probably very little. After all, unless the method call itself did bugger-all work, the time to create the method object is probably insignificant. But it's a possible optimization. An interesting alternative (if it's not already being done) might be to maintain a limited free-list of method objects, removing the need to allocate memory for one before filling it in with data. It is already done. Some stats were posted recently to python-dev, and (if I read them correctly) method objects are among the free-list types. So the actual memory (de)allocations are optimized, and all that's left is setting a couple of pointers to select an object and a function. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
[issue24202] Multiprocessing Pool not working for userdefined function
Changes by Davin Potts pyt...@discontinuity.net: -- nosy: +davin type: crash - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24202 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24202] Multiprocessing Pool not working for userdefined function
Paul Moore added the comment:
OK, if it's not reproducible in a standalone script, I'll close this as it's
expected behaviour.
Correcting the typo in your script (__name__ == __main__), I ran it and it
worked as expected on my system:
type multi.py
import multiprocessing
import numpy
def f(x):
return x*x
if __name__ == __main__:
p= multiprocessing.Pool(5)
print(p.map(numpy.sqrt,[1,2,3,4]))
print(p.map(f,[1,2,3,4]))
PS 15:05 {00:00.089} C:\Work\Scratch
py .\multi.py
[1.0, 1.4142135623730951, 1.7320508075688772, 2.0]
[1, 4, 9, 16]
I'm not sure why you weren't getting output, but it doesn't look like a Python
issue.
--
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24202
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24195] Add `Executor.filter` to concurrent.futures
Ethan Furman added the comment: Short History: = (Ram Rachum) What do you think about adding a method: `Executor.filter`? I was using something like this: my_things = [thing for thing in things if some_condition(thing)] But the problem was that `some_condition` took a long time to run waiting on I/O, which is a great candidate for parallelizing with ThreadPoolExecutor. I made it work using `Executor.map` and some improvizing, but it would be nicer if I could do: with concurrent.futures.ThreadPoolExecutor(100) as executor: my_things = executor.filter(some_condition, things) And have the condition run in parallel on all the threads. (Nick Coughlan) I think this is sufficiently tricky to get right that it's worth adding filter() as a parallel to the existing map() API. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24195 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24162] [2.7 regression] test_asynchat test failure on i586-linux-gnu
Roundup Robot added the comment: New changeset 7adfc99103d2 by Benjamin Peterson in branch '2.7': deque is not varsized, so using Py_SIZE is nonsensical (closes #24162) https://hg.python.org/cpython/rev/7adfc99103d2 -- nosy: +python-dev resolution: - fixed stage: - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24162 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24114] ctypes.utils uninitialized variable 'path'
Martin Panter added the comment: I fail to see how this patch does anything. What is the actual error or traceback? Perhaps it is for “paths” (plural)? -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24114 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23840] tokenize.open() leaks an open binary file on TextIOWrapper error
STINNER Victor added the comment: You should see the new file in the next 30 minutes. I don't see the new file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23840 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call IOW, two dict lookups plus an object construction for each method call. Marko -- https://mail.python.org/mailman/listinfo/python-list
[issue24064] Make the property doctstring writeable
Roundup Robot added the comment: New changeset 5262dd507ee5 by Raymond Hettinger in branch 'default': Issue #24064: Docuement that oroperty docstrings are now writeable. https://hg.python.org/cpython/rev/5262dd507ee5 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24064 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On 15/05/2015 23:44, Marko Rauhamaa wrote: BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call IOW, two dict lookups plus an object construction for each method call. Marko As a picture paints a thousand words is anybody aware of a site or sites that show this diagramatically, as I think I and possibly others would find it far easier to grasp. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
[issue24204] string.strip() documentation is misleading
R. David Murray added the comment:
Indeed, the sentence that confused you was added because people were
continually confused by the fact that doing:
'foo.boo'.strip('.boo')
would result in 'f' rather than 'foo'.
We would welcome an improvement, but apparently this is very hard to make clear.
Repeating the mississippi example is probably a good idea.
--
nosy: +r.david.murray
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24204
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24205] signature.bind error messages are sub-optimal
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24205 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On 2015-05-16 01:43, BartC wrote: On 15/05/2015 23:44, Marko Rauhamaa wrote: BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call IOW, two dict lookups plus an object construction for each method call. OK, I didn't know that objects have their own set of attributes that are distinct from the class they belong to. I really ought to learn more Python!. (Yet, I have this crazy urge now to create my own bytecode interpreter for, if not exactly Python itself, then an equivalent language. Just to see if I can do any better than CPython, given the same language restraints. Although I'm hampered a little by not knowing Python well enough. Nor OOP, but those are minor details... Anyway it sounds more fun than trying to decipher the layers of macros and conditional code that appear to be the CPython sources.) IOW, two dict lookups plus an object construction for each method call. I suppose in many cases an object will have no attributes of its own, and so it can rapidly bypass the first lookup. I don't understand the need for an object creation (to represent A.B so that it can call it?) but perhaps such an object can already exist, prepared ready for use. It's possible to do: f = A.B ... f() so it's necessary to have an object for A.B. The question is how much you would gain from optimising A.B() as a special case (increase in speed vs increase in complexity). -- https://mail.python.org/mailman/listinfo/python-list
نتيجة الشهادة الابتدائية 2015 الترم الثانى
نتيجة الشهادة الابتدائية 2015 الترم الثانى https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية 2015 برقم الجلوس جميع المحافظات https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة القاهرة 2015 برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة الاسكندرية 2015 https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl استعلم عن ارقام جلوس الثانوية العامة 2015 https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة البحر الاحمر 2015 الترم الثانى برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة الاقصر 2015 الترم الثانى برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة قنا الترم الثانى 2015 برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الشهادة الابتدائية محافظة كفر الشيخ الترم الثانى 2015 برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl طريقة الحصول على رقم الجلوس الخاص بك فى امتحانات الثانوية العامة 2015 https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl نتيجة الصف السادس الابتدائى الترم الثانى 2015 برقم الجلوس https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl الإجابة النموذجية لامتحان اللغة العربية محافظة الجيزة للصف السادس الابتدائى 2015 https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl الإجابة النموذجية لامتحان اللغة العربية محافظة القليوبية للشهادة الابتدائية الترم الثانى 2015 https://plus.google.com/u/0/106736449589858805776 https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl أهم توقعات ليلة الامتحان فى الدراسات الإجتماعية الشهادة الإعدادية
Re: Building CPython
On 5/15/2015 5:54 PM, BartC wrote: What /is/ a method lookup? Is it when you have this: A.B() This is parsed as (A.B)() and need to find whether the expression A (or its class or type) has a name B associated with it? Yes. Dotted names imply an attribute lookup. (And it then needs to check whether B is something that can be called.) The object resulting from the attribute lookup, A.B (not B exactly), is called in a separate operation (with a separate bytecode). It depends on the object having a .__call__ method. The .__call__ method is *executed* (rather than *called*, which would lead to infinite regress). -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
[issue24205] signature.bind error messages are sub-optimal
Changes by Ethan Furman et...@stoneleaf.us: -- nosy: +ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24205 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Survey -- Move To Trash function in Python?
On Friday, May 15, 2015 at 11:27:18 AM UTC-7, rand...@fastmail.us wrote: On Fri, May 15, 2015, at 00:25, Chris Angelico wrote: The main thing is that trashing invites the system to delete the file at its leisure, I've never seen a system whose trash can emptied itself without user intervention. Windows won't empty the entire trash without user intervention, but is IS configured by default to only use up to a specific amount of hard drive space. Once the trash reaches its capacity, it'll start removing files from it to make room for more. -- https://mail.python.org/mailman/listinfo/python-list
[issue24192] unexpected system error with pep420 style namespace packages
Changes by Eric Snow ericsnowcurren...@gmail.com: -- assignee: - eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24192 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9694] argparse required arguments displayed under optional arguments
Martin Panter added the comment: Is there any interest in my or Ryan’s patches, which change the default heading away from “optional arguments”? Changing the default is my preferred fix, but if others don’t like it (e.g. compatibility concerns), I am happy to work on a documentation patch according to https://bugs.python.org/issue9694#msg132327. I don’t see how adding an extra help groups API is very helpful though. The problem here is that the default help group headings are wrong (or at least misleading to many people). -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9694 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24192] unexpected system error with pep420 style namespace packages
Eric Snow added the comment: Here's a fix. If I don't hear from anyone right away I'll push it in a few hours (or tomorrow morning). -- keywords: +patch stage: - patch review Added file: http://bugs.python.org/file39386/issue24192.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24192 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
BartC wrote: For example, there is a /specific/ byte-code called BINARY_ADD, which then proceeds to call a /generic/ binary-op handler! This throws away the advantage of knowing at byte-code generation time exactly which operation is needed. While inlining the binary-op handling might give you a slightly shorter code path, it wouldn't necessarily speed anything up. It's possible, for example, that the shared binary-op handler fits in the instruction cache, but the various inlined copies of it don't, leading to a slowdown. The only way to be sure about things like that is to try them and measure. The days when you could predict the speed of a program just by counting the number of instructions executed are long gone. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Minimising stack trace
On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote: Cecil Westerhof wrote: While playing with recursion I get: RuntimeError: maximum recursion depth exceeded in comparison But then I get a very long stack trace. Is there a way to make this a lot shorter. Now I ‘lose’ interesting information because of the length of the stack trace. http://www.catb.org/~esr/faqs/smart-questions.html There ought to be a website that explains how to give smart answers. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
[issue24205] signature.bind error messages are sub-optimal
New submission from R. David Murray:
I have an application where I'm calling a handler function with passed in
arguments. I want to generate an error if the handler is called with the wrong
arguments. I can't just catch TypeError since a TypeError could easily result
from some programming error in the handler, rather than an error in the calling
args. So, doing this seems obvious:
sig = signature(handler)
try:
bound = sig.bind(message, payload, *args, **kw)
except TypeError as exc:
print(Invalid handler call: {}.format(str(exc))
handler(*bound.args, **bound.kwargs)
Now, suppose I have a function like:
def foo(message, payload, anarg, akeyword='bar'):
pass
If I call it directly with an invalid keyword argument I get:
foo(1, 2, 3, badword=7)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: foo() got an unexpected keyword argument 'badword'
However, bind gives me:
sig.bind(1, 2, 3, badword=7)
Traceback (most recent call last):
File stdin, line 1, in module
File /home/rdmurray/python/p35/Lib/inspect.py, line 2818, in bind
return args[0]._bind(args[1:], kwargs)
File /home/rdmurray/python/p35/Lib/inspect.py, line 2809, in _bind
raise TypeError('too many keyword arguments')
TypeError: too many keyword arguments
Similarly, for a missing argument I get:
foo(1, 2)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: foo() missing 1 required positional argument: 'anarg'
While bind gives:
sig.bind(1, 2)
Traceback (most recent call last):
File stdin, line 1, in module
File /home/rdmurray/python/p35/Lib/inspect.py, line 2818, in bind
return args[0]._bind(args[1:], kwargs)
File /home/rdmurray/python/p35/Lib/inspect.py, line 2737, in _bind
raise TypeError(msg) from None
TypeError: 'anarg' parameter lacking default value
So, using this to replace catching the TypeError from incorrectly calling a
function does not work. Nor are the messages in fact accurate. Is there any
chance we could make bind's error handling work like regular function binding?
That's certainly what I expected would happen!
--
components: Library (Lib)
messages: 243287
nosy: r.david.murray, yselivanov
priority: normal
severity: normal
status: open
title: signature.bind error messages are sub-optimal
type: behavior
versions: Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24205
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Python: How to find out values of all feasible x under constraints.
On 15/05/2015 13:35, Mark Lawrence wrote: On 15/05/2015 04:58, Xiang Zhang wrote: Dear all, I am writing a code using Python now. I want to know how to find out values of all feasible x under constraints. x = [x_1, x_2, x_3,..., x_10] constraints: x_i = 0,1,2,3 or 4, where i=1,2,10 x_1 + x_2 + x_3 +...+x_10 = 15 How to find out all the feasible solutions x (domain of x) using python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the code? Any hint or help would be highly appreciated! Sincerely, Xiang Zhang There are several constraint libraries on pypi if you don't want to roll your own. See for example https://pypi.python.org/pypi/python-constraint/1.2 Forgot to mention this http://numberjack.ucc.ie/ Maybe of general interest see also http://www.csplib.org/ http://www.hakank.org/constraint_programming_blog/ -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On 15/05/2015 23:44, Marko Rauhamaa wrote: BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call IOW, two dict lookups plus an object construction for each method call. OK, I didn't know that objects have their own set of attributes that are distinct from the class they belong to. I really ought to learn more Python!. (Yet, I have this crazy urge now to create my own bytecode interpreter for, if not exactly Python itself, then an equivalent language. Just to see if I can do any better than CPython, given the same language restraints. Although I'm hampered a little by not knowing Python well enough. Nor OOP, but those are minor details... Anyway it sounds more fun than trying to decipher the layers of macros and conditional code that appear to be the CPython sources.) IOW, two dict lookups plus an object construction for each method call. I suppose in many cases an object will have no attributes of its own, and so it can rapidly bypass the first lookup. I don't understand the need for an object creation (to represent A.B so that it can call it?) but perhaps such an object can already exist, prepared ready for use. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On 15/05/2015 09:59, Marko Rauhamaa wrote: The path from decoding a bytecode to the C code that implements it can be rather convoluted, but there are reasons for each of the complications -- mainly to do with supporting the ability to override operators with C and/or Python code. If you removed those abilities, the implemention would be simpler, and possibly faster. But then the language wouldn't be Python any more. I agree that Python's raison-d'être is its dynamism and expressive power. It definitely shouldn't be sacrificed for performance. However, in some respects, Python might be going overboard with its dynamism; are all those dunder methods really needed? Must false be defined so broadly? Must a method lookup necessarily involve object creation? What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) (And I guess Python's classes come into play so if B is not part of A's class then it might be part of some base-class. I can see that it can get complicated, but I don't use OO so can't speculate further.) (In the language whose implementation I'm comparing with CPython's, it doesn't have classes. A.B() can still appear, but .B will need to be an attribute that the (bytecode) compiler already knows from a prior definition (usually, some struct or record if A is an expression). If there is only one .B it knows, then a simple check that A is the correct type is all that is needed. Otherwise a runtime search through all .Bs (and a compact table will have been set up for this) is needed to find the .B that matches A's type. But this is all still pretty quick. In this context, A.B will need to be some function pointer, and A.B() will call it.) -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
[issue17620] Python interactive console doesn't use sys.stdin for input
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: -haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17620 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24192] unexpected system error with pep420 style namespace packages
Changes by Brett Cannon br...@python.org: -- priority: normal - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24192 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Trying to build Python
Op Friday 15 May 2015 13:16 CEST schreef Chris Angelico: On Fri, May 15, 2015 at 9:02 PM, Rustom Mody rustompm...@gmail.com wrote: On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote: I am trying to build Python 3 on a Debian system. It is successful, but a few things where missing. Partly I solved it, but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl _gdbm What do I need to do to get those resolved also? $ aptitude build-dep python3 should fetch the requirements of python3 It should, but it did not. I had to install tcl-dev and tk-dev and then still got the above missing dependencies. And if not, try grabbing some development libraries: $ sudo apt-get install libbz2-dev libsqlite3-dev libreadline-dev libssl-dev libgdbm-dev That did the trick. :-D I'm not sure about the _dbm module, not sure what it needs. I get no complaints anymore, so: problem solved. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: How much time would it save? Probably very little. After all, unless the method call itself did bugger-all work, the time to create the method object is probably insignificant. But it's a possible optimization. An interesting alternative (if it's not already being done) might be to maintain a limited free-list of method objects, removing the need to allocate memory for one before filling it in with data. -- https://mail.python.org/mailman/listinfo/python-list
[issue23699] Add a macro to ease writing rich comparisons
Raymond Hettinger added the comment: You don't need a PEP. If Barry and Marc-Andre want this to go forward, I won't hold it back. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23699 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24201] _winreg PyHKEY Type Confusion
eryksun added the comment: wherein pointers can be passed directly in place of PyHKEY instances e.g. _winreg.QueryValue(0x41414141, ) If a debugger is attached you see the first-chance exception for the access violation. Normally the registry function simply returns ERROR_INVALID_HANDLE (6), which gets raised as an OSError in Python. The call took the RPC path because the low bit (1) marks a remote handle, which is actually a pointer to a data structure. RPCRT4!NDRCContextBinding looks for a signature (0xFEDCBA98) to validate this structure. In this case the attempt raised an access violation, which gets handled by raising another exception with the exception code set to ERROR_INVALID_HANDLE. The same exception gets raised if it can't validate the handle. Subsequently this exception is handled by calling RPCRT4!NdrClientMapCommFault to map the code to a return value. For example (x64 ISA): _winreg.QueryInfoKey(0x41414141) (a2c.828): Access violation - code c005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. RPCRT4!NDRCContextBinding+0x4: 07fe`fefca6e4 81790898badcfe cmp dword ptr [rcx+8],0FEDCBA98h ds:`41414148= 0:000 gN (a2c.828): Unknown exception - code 0006 (first chance) Breakpoint 0 hit RPCRT4!NdrClientMapCommFault: 07fe`ff05f010 fff3pushrbx 0:000 kc 8 Call Site RPCRT4!NdrClientMapCommFault RPCRT4!NdrpClientCall3 RPCRT4!NdrClientCall3 ADVAPI32!SafeBaseRegQueryInfoKey ADVAPI32!RemoteRegQueryInfoKeyWrapper kernel32!TlsGetValue ADVAPI32!RegQueryInfoKeyAStub python27!PyQueryInfoKey The exception code is passed in register r8 and gets assigned to the address in r9: 0:000 r r8, r9 r8=0006 r9=0021f1d8 0:000 dd 21f1d8 l1 `0021f1d8 0:000 pt RPCRT4!NdrClientMapCommFault+0x80: 07fe`ff05f080 c3 ret 0:000 dd 21f1d8 l1 `0021f1d8 0006 This return value gets passed back up the call stack: 0:000 gu; pt; r rax rax=0006 0:000 gu; pt; r rax rax=0006 0:000 gu; pt; r rax rax=0006 0:000 gu; pt; r rax rax=0006 0:000 gu; pt; r rax rax=0006 0:000 r rax=0006 rbx=00e1cda0 rcx= rdx= rsi= rdi=1e1027b0 rip=779ba204 rsp=0021f9d8 rbp=00eb61c8 r8=0021f1d8 r9= r10=0021f1d8 r11=0021f8b0 r12=00e1cda0 r13=00807bb0 r14=1e2b3210 r15=00eb7060 iopl=0 nv up ei pl nz na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=0206 kernel32!RegQueryInfoKeyA+0x364: `779ba204 c3 ret Until finally getting raised as a Python exception: 0:000 g Traceback (most recent call last): File stdin, line 1, in module WindowsError: [Error 6] The handle is invalid The odds are extremely low that someone will pass in an integer address that's flagged as a remote handle (ends in 1) and is a valid, mapped address that contains the RPC handle signature. Even then, it won't reference an actual proxy handle for a remote registry, so it'll just fail farther along the chain. I'm sure if a feature exists that someone, somewhere depends on it, so I don't see a reason to change this unless there's a real problem here. Is there a specific technical or security problem that you see here? -- nosy: +eryksun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24201 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17305] IDNA2008 encoding missing
Changes by Christian Heimes li...@cheimes.de: -- nosy: +christian.heimes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Building CPython
On Fri, May 15, 2015 at 9:00 AM, Ian Kelly ian.g.ke...@gmail.com wrote: On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: How much time would it save? Probably very little. After all, unless the method call itself did bugger-all work, the time to create the method object is probably insignificant. But it's a possible optimization. An interesting alternative (if it's not already being done) might be to maintain a limited free-list of method objects, removing the need to allocate memory for one before filling it in with data. Looks like it is already being done: https://hg.python.org/cpython/file/e7c7431f91b2/Objects/methodobject.c#l7 -- https://mail.python.org/mailman/listinfo/python-list
Re: Minimising stack trace
On Friday, May 15, 2015 at 9:33:54 PM UTC-4, Chris Angelico wrote: On Sat, May 16, 2015 at 11:24 AM, Ned Batchelder n...@nedbatchelder.com wrote: On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote: On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote: Cecil Westerhof wrote: While playing with recursion I get: RuntimeError: maximum recursion depth exceeded in comparison But then I get a very long stack trace. Is there a way to make this a lot shorter. Now I 'lose' interesting information because of the length of the stack trace. http://www.catb.org/~esr/faqs/smart-questions.html There ought to be a website that explains how to give smart answers. Or one that explains how to be kind to those that don't know the ropes yet. Though pointing impatient experts at a website probably won't get them to change any more than brusquely pointing newbs at a website will. You mean like this? http://www.catb.org/~esr/faqs/smart-questions.html#idp64834912 (Not sure whether you knew and were hinting, or didn't know that that's explicitly mentioned.) I didn't realize that was in there. :) It's a good demonstration that we can all improve in various ways. Being right isn't the highest ideal, there are other things to aspire to as well. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Sat, 16 May 2015 09:27 am, Mark Lawrence wrote: On 15/05/2015 23:44, Marko Rauhamaa wrote: BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) It's not as inefficient as you may think. Dicts are hash tables, and hash tables are a standard computer science data structure for performing very fast searches at constant (or near constant) time. The dict is basically an array of pointers to (key, value). To look a name up in the dict, you hash the string which gives you an index into the array, then look at that position. If it is blank, you know there is no match. If it points to a string, you compare that string to your string. If they are equal, then you have a match. If they aren't equal, you have a collision, and you have to look elsewhere (details differ) but typically you don't end up looking in more than one or two positions. So all pretty fast, and close enough to constant time. To speed things up even further, I think that the hash value is cached with the string, so it only needs to be calculated the first time. That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call It's the other way around. The function object already exists: you created it by writing `def method(self, *args): ... ` inside the class body. def always makes a function. It's the *method* object which is created on the fly, delegating to the function. IOW, two dict lookups plus an object construction for each method call. Marko As a picture paints a thousand words is anybody aware of a site or sites that show this diagramatically, as I think I and possibly others would find it far easier to grasp. No I'm not aware of any such site, but I can try to make it more obvious with an example. Suppose we have a hierarchy of classes, starting from the root of the hierarchy (object) to a specific instance: class Animal(object): pass class Mammal(Animal): pass class Dog(Mammal): def bark(self): ... laddy = Dog() We then look up a method: laddy.bark() In a non-dynamic language like Java, the compiler knows exactly where bark is defined (in the Dog class) and can call it directly. In dynamic languages like Python, the compiler can't be sure that bark hasn't been shadowed or overridden at runtime, so it has to search for the first match found. Simplified: * Does laddy.__dict__ contain the key bark? If so, we have a match. * For each class in the MRO (Method Resolution Order), namely [Dog, Mammal, Animal, object], does the class __dict__ contain the key bark? If so, we have a match. * Do any of those classes in the MRO have a __getattr__ method? If so, then try calling __getattr__(bark) and see if it returns a match. * Give up and raise AttributeError. [Aside: some of the things I haven't covered: __slots__, __getattribute__, how metaclasses can affect this.] In the case of laddy.bark, the matching attribute is found as Dog.__dict__['bark']: temp = Dog.__bark__['bark'] # laddy.bark, is a function At this point, the descriptor protocol applies. You can ignore this part if you like, and just pretend that laddy.bark returns a method instead of a function, but if you want to know what actually happens in all it's gory details, it is something like this (again, simplified): * Does the attribute have a __get__ method? If not, then we just use the object as-is, with no changes. * But if it does have a __get__ method, then it is a descriptor and we call the __get__ method to get the object we actually use. Since functions are descriptors, we get this: temp = temp.__get__(laddy, Dog) # returns a method object and finally we can call the method: temp() # laddy.bark() None of these individual operations are particularly expensive, nor are there a lot of them. For a typical instance, the MRO usually contains only two or three classes, and __dict__ lookups are fast. Nevertheless, even though each method lookup is individually *almost* as fast as the sort of pre-compiled all-but-instantaneous access which Java can do, it all adds up. So in Java, a long chain of dots: foo.bar.baz.foobar.spam.eggs.cheese can be resolved at compile-time, and takes no more time than
Re: pip grabs tar.gz file instead of whl?
On 16/05/2015 02:26, Chris Angelico wrote: On Sat, May 16, 2015 at 7:00 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The same file installed perfectly when downloaded and installed in two steps. Whether this is simply a known bug with zipfile handling, pip itself, a combination of both or what I've no idea. I find it slightly irritating that a tool recommended by the Python Packaging Authority behaves in such a way, but then I didn't have to dip into my pocket to pay for it and certainly won't lose any sleep over it as there's such a simple work around. The way I see it, pip is great for handling the most common case where you just want to name a package and say go fetch, but if you want to override its decisions, you should use the lower-level facilities eg manual downloading and setup.py. It's like with Debian packages: I can type sudo apt-get install blah and it'll run off and grab it, check its signatures, make sure everything's right, and then install it; but if I want to install something from a different location, the best way is usually to download it manually, do my own checking, and then sudo dpkg -i blah.deb to actually install it - no apt-get involvement at all. This shouldn't normally be a problem; you don't *have* to use pip here, you just want to end up with the package properly installed. ChrisA Being on Windows, as I said at the beginning of the thread, the biggest problem is that setup.py can't find VS if there is no whl file to install. Hence it is far easier to get the binaries from elsewhere. Hopefully this problem will disappear in the future as the whl standard becomes prevelant. As for sudo I always thought that was a type of Japanese wrestling :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On Sat, May 16, 2015 at 11:55 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: but in Python's case it has to be resolved at run-time, so if you care about speed, you should try to avoid long chains of dots in performance critical loops. E.g. instead of: for x in sequence: foo.bar.baz.foobar.spam.eggs.cheese(x) you can write: cheese = foo.bar.baz.foobar.spam.eggs.cheese for x in sequence: cheese(x) Like all advice, of course, this should not be taken on its own. Some code tries to avoid long chains of dots by adding wrapper methods: for x in sequence: foo.cheese(x) where foo.cheese() delegates to self.bar.cheese() and so on down the line. That, of course, will be far FAR slower, in pretty much any language (unless the compiler can in-line the code completely, in which case it's effectively being optimized down to the original anyway); the dots aren't as bad as you might think :) Just always remember the one cardinal rule of optimization: MEASURE FIRST. You have no idea how slow something is until you measure it. (I'm not addressing my comments to Steven here, who I'm fairly sure is aware of all of this(!), but it's his post that gave the example that I'm quoting.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Building CPython
On 16/05/2015 02:55, Steven D'Aprano wrote: On Sat, 16 May 2015 09:27 am, Mark Lawrence wrote: On 15/05/2015 23:44, Marko Rauhamaa wrote: BartC b...@freeuk.com: What /is/ a method lookup? Is it when you have this: A.B() and need to find whether the expression A (or its class or type) has a name B associated with it? (And it then needs to check whether B is something that can be called.) If so, does that have to be done using Python's Dict mechanism? (Ie. searching for a key 'B' by name and seeing if the object associated with it is a method. That does not sound efficient.) It's not as inefficient as you may think. Dicts are hash tables, and hash tables are a standard computer science data structure for performing very fast searches at constant (or near constant) time. The dict is basically an array of pointers to (key, value). To look a name up in the dict, you hash the string which gives you an index into the array, then look at that position. If it is blank, you know there is no match. If it points to a string, you compare that string to your string. If they are equal, then you have a match. If they aren't equal, you have a collision, and you have to look elsewhere (details differ) but typically you don't end up looking in more than one or two positions. So all pretty fast, and close enough to constant time. To speed things up even further, I think that the hash value is cached with the string, so it only needs to be calculated the first time. That is a general feature among high-level programming languages. In Python, it is even more complicated: * first the object's dict is looked up for the method name * if the method is not found (it usually isn't), the dict of the object's class is consulted * if the method is found (it usually is), a function object is instantiated that delegates to the class's method and embeds a self reference to the object to the call It's the other way around. The function object already exists: you created it by writing `def method(self, *args): ... ` inside the class body. def always makes a function. It's the *method* object which is created on the fly, delegating to the function. IOW, two dict lookups plus an object construction for each method call. Marko As a picture paints a thousand words is anybody aware of a site or sites that show this diagramatically, as I think I and possibly others would find it far easier to grasp. No I'm not aware of any such site, but I can try to make it more obvious with an example. Suppose we have a hierarchy of classes, starting from the root of the hierarchy (object) to a specific instance: class Animal(object): pass class Mammal(Animal): pass class Dog(Mammal): def bark(self): ... laddy = Dog() We then look up a method: laddy.bark() In a non-dynamic language like Java, the compiler knows exactly where bark is defined (in the Dog class) and can call it directly. In dynamic languages like Python, the compiler can't be sure that bark hasn't been shadowed or overridden at runtime, so it has to search for the first match found. Simplified: * Does laddy.__dict__ contain the key bark? If so, we have a match. * For each class in the MRO (Method Resolution Order), namely [Dog, Mammal, Animal, object], does the class __dict__ contain the key bark? If so, we have a match. * Do any of those classes in the MRO have a __getattr__ method? If so, then try calling __getattr__(bark) and see if it returns a match. * Give up and raise AttributeError. [Aside: some of the things I haven't covered: __slots__, __getattribute__, how metaclasses can affect this.] In the case of laddy.bark, the matching attribute is found as Dog.__dict__['bark']: temp = Dog.__bark__['bark'] # laddy.bark, is a function At this point, the descriptor protocol applies. You can ignore this part if you like, and just pretend that laddy.bark returns a method instead of a function, but if you want to know what actually happens in all it's gory details, it is something like this (again, simplified): * Does the attribute have a __get__ method? If not, then we just use the object as-is, with no changes. * But if it does have a __get__ method, then it is a descriptor and we call the __get__ method to get the object we actually use. Since functions are descriptors, we get this: temp = temp.__get__(laddy, Dog) # returns a method object and finally we can call the method: temp() # laddy.bark() None of these individual operations are particularly expensive, nor are there a lot of them. For a typical instance, the MRO usually contains only two or three classes, and __dict__ lookups are fast. Nevertheless, even though each method lookup is individually *almost* as fast as the sort of pre-compiled all-but-instantaneous access which Java can do, it all adds up. So in Java, a long chain of dots: foo.bar.baz.foobar.spam.eggs.cheese can be resolved at
How to deploy a custom common module?
While working on one python script (test.py), I developed some functions that I will probably need in my future projects, so I decided to put such functions in another python file (cmn_funcs.py). So in my test.py there is import cmn_funcs in order to use common functions and everything works well. When I deploy test.py on another computer, I put (rsync) both test.py and cmn_funcs.py in the same remote directory. If I create another python project (test2.py) in new directory, that needs common functions, what should I do with cmn_funcs.py? Cmn_funcs.py should be in some shared directory accessible by my every current and future python project. Furthermore, I would like to preserve simplicity of using rsync for upgrading these three python files on remote computers. How to deploy custom a custom common module? Where to put it? How to use it? How to upgrade it? -- https://mail.python.org/mailman/listinfo/python-list
Re: Minimising stack trace
On Sat, May 16, 2015 at 11:24 AM, Ned Batchelder n...@nedbatchelder.com wrote: On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote: On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote: Cecil Westerhof wrote: While playing with recursion I get: RuntimeError: maximum recursion depth exceeded in comparison But then I get a very long stack trace. Is there a way to make this a lot shorter. Now I 'lose' interesting information because of the length of the stack trace. http://www.catb.org/~esr/faqs/smart-questions.html There ought to be a website that explains how to give smart answers. Or one that explains how to be kind to those that don't know the ropes yet. Though pointing impatient experts at a website probably won't get them to change any more than brusquely pointing newbs at a website will. You mean like this? http://www.catb.org/~esr/faqs/smart-questions.html#idp64834912 (Not sure whether you knew and were hinting, or didn't know that that's explicitly mentioned.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
[issue24201] _winreg PyHKEY Type Confusion
JohnLeitch added the comment: Thank you for taking the time to peruse my report and explain the behavior I observed. My understanding of Windows RPC internals is lacking, and perhaps I jumped the gun upon catching an AV while fuzzing. That said, after poking around to better understand the matter, I discovered a few things: 1) There are code paths where it is possible to trigger an unhandled access violation: 0:000 g (11a0.d54): Access violation - code c005 (!!! second chance !!!) eax=60dad396 ebx= ecx= edx= esi=00a0 edi=00a0 eip=776f1037 esp=0027f790 ebp=0027f80c iopl=0 nv up ei pl nz ac pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010216 ntdll!RtlAllocateHeap+0x17: 776f1037 8b4344 mov eax,dword ptr [ebx+44h] ds:002b:0044=?? ?? 0:000 k ChildEBP RetAddr 0027f80c 75471731 ntdll!RtlAllocateHeap+0x17 0027f82c 75479616 RPCRT4!AllocWrapper+0x2d 0027f840 75479791 RPCRT4!ThreadSelfHelper+0x16 0027f848 754f6b2a RPCRT4!ThreadSelf+0x18 0027fc74 753a5d94 RPCRT4!NdrClientCall2+0x13a 0027fc8c 7539f48b ADVAPI32!BaseRegGetVersion+0x24 0027fce4 7538dfce ADVAPI32!RegDeleteKeyW+0x14aeb 0027fd00 1e0de533 ADVAPI32!RegDeleteKeyA+0x2e 0027fd18 1e0aafd7 python27!PyDeleteKey+0x53 0027fd30 1e0edd10 python27!PyCFunction_Call+0x47 0027fd5c 1e0f017a python27!call_function+0x2b0 0027fdcc 1e0f1150 python27!PyEval_EvalFrameEx+0x239a 0027fe00 1e0f11b2 python27!PyEval_EvalCodeEx+0x690 0027fe2c 1e11707a python27!PyEval_EvalCode+0x22 0027fe44 1e1181c5 python27!run_mod+0x2a 0027fe64 1e118760 python27!PyRun_FileExFlags+0x75 0027fea4 1e1190d9 python27!PyRun_SimpleFileExFlags+0x190 0027fec0 1e038d35 python27!PyRun_AnyFileExFlags+0x59 0027ff3c 1d00116d python27!Py_Main+0x965 0027ff80 75967c04 python!__tmainCRTStartup+0x10f 0027ff94 7770ad1f KERNEL32!BaseThreadInitThunk+0x24 0027ffdc 7770acea ntdll!__RtlUserThreadStart+0x2f 0027ffec ntdll!_RtlUserThreadStart+0x1b 0:000 This doesn't appear to be outright exploitable for anything beyond DoS, but it does crash the process. 2) Assuming attacker control of the hkey parameter to a _winreg call, I believe it would be possible to leverage the RPC signature check to disclose the location of valid memory such as the RPC module itself, thereby bypassing ASLR. import _winreg _winreg.DeleteKey(0x75469AF1, '') Traceback (most recent call last): File pyshell#13, line 1, in module _winreg.DeleteKey(0x75469AF1, '') WindowsError: [Error 6] The handle is invalid _winreg.DeleteKey(0x75469AF3, '') RESTART 0:000 dd 0x75469AF3 75469af3 98047881 0ffedcba 01d9a085 fc45c700 75469b03 fffe 8fe8008b c2a2 9094 75469b13 90909090 fe90 00ff d400 75469b23 00ff fe00 49d901ff 49d92575 75469b33 06f76875 b2e8 8b000161 47c7e845 75469b43 003c 50478900 00a661e9 09be0f00 75469b53 00a649e9 06f76800 8ee8 68000161 75469b63 06e6 016184e8 90909000 499e5190 0:000 !address 0x75469AF3 Usage: Image Base Address: 75451000 End Address:754fa000 Region Size:000a9000 State: 1000MEM_COMMIT Protect:0020PAGE_EXECUTE_READ Type: 0100MEM_IMAGE Allocation Base:7545 Allocation Protect: 0080PAGE_EXECUTE_WRITECOPY Image Path: C:\WINDOWS\SysWOW64\RPCRT4.dll Module Name:RPCRT4 Loaded Image Name: C:\WINDOWS\SYSTEM32\RPCRT4.dll Mapped Image Name: More info: lmv m RPCRT4 More info: !lmi RPCRT4 More info: ln 0x75469af3 More info: !dh 0x7545 3) Finally, I still suspect it may be possible to achieve memory corruption with this bug, but cannot verify without a better understanding of the structures at play and further analysis. The hypothetical attack goes like this: a) The attacker sprays memory with carefully constructed buffers containing the expected magic numbers at the correct offsets. b) Once memory has been sufficiently sprayed, the attacker triggers the bug with an hkey value that is actually an address predicted to be one of the sprayed structures. c) While working with the attacker controlled buffer, RPC inadvertently corrupts memory. Step C is, of course, dependent on what what fields are available in the structure, and what RPC does with them. Unfortunately I can't find any relevant documentation or code, but in my testing I was able to force different code paths. Some look fruitful, but determining whether they're reachable would be costly time-wise, and it's a bit of a moot point because we're talking about Microsoft's internal implementation, which could change at any point, altering exploitability. Given that, I'd say it's best to err on the side of caution, and assume corruption is possible with the right primitives. --
Re: Minimising stack trace
On Saturday, May 16, 2015 at 8:22:05 AM UTC+5:30, Rustom Mody wrote: On Saturday, May 16, 2015 at 6:54:23 AM UTC+5:30, Ned Batchelder wrote: On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote: On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote: Cecil Westerhof wrote: While playing with recursion I get: RuntimeError: maximum recursion depth exceeded in comparison But then I get a very long stack trace. Is there a way to make this a lot shorter. Now I 'lose' interesting information because of the length of the stack trace. http://www.catb.org/~esr/faqs/smart-questions.html There ought to be a website that explains how to give smart answers. Or one that explains how to be kind to those that don't know the ropes yet. Though pointing impatient experts at a website probably won't get them to change any more than brusquely pointing newbs at a website will. How about a 'answerer-quota' ? « Once for every 20 useful answers you are allowed to vent on any subject of your choice... - Stupid questions (and point to catb's smart questions) - Google groups - Top posting - Html mail » Forgot the most important: Using print's -- https://mail.python.org/mailman/listinfo/python-list
[issue23184] Unused imports, variables, file in IDLE
Changes by Terry J. Reedy tjre...@udel.edu: -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23184] Unused imports, variables, file in IDLE
Roundup Robot added the comment: New changeset 8875d7c6a99d by Terry Jan Reedy in branch '2.7': Issue #23184: idle tests, remove unused names and imports. https://hg.python.org/cpython/rev/8875d7c6a99d New changeset 526ce81f700d by Terry Jan Reedy in branch '3.4': Issue #23184: idle tests, remove unused names and imports. https://hg.python.org/cpython/rev/526ce81f700d New changeset 2a56eb5c08cc by Terry Jan Reedy in branch 'default': Merge 3.4 #23184: idle tests, remove unused names and imports. https://hg.python.org/cpython/rev/2a56eb5c08cc -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Instead of deciding between Python or Lisp for a programming intro course...What about an intro course that uses *BOTH*? Good idea?
On Friday, May 15, 2015 at 10:36:45 PM UTC-5, Rustom Mody wrote: On Saturday, May 16, 2015 at 8:56:00 AM UTC+5:30, zipher wrote: On Wednesday, May 13, 2015 at 9:36:27 PM UTC-5, Mark Lawrence wrote: On 14/05/2015 02:40, Steven D'Aprano wrote: On Thu, 14 May 2015 04:07 am, zipher wrote: No, you haven't understood, padawan. Lambda *is* the function, not it's definition. Perhaps you will understand what I mean by that, perhaps you won't. It's subtle. Subtle like a kick to the head. From the very first drivel that he posted on python ideas just over two years ago, he's shown that he's incapable of any logical thought relating to computing, in just the same way that the RUE has never posted anything logical about the FSR. Please can we stop feeding him. The truth is quite the opposite. There is something very strange going on around the nets. People are very confused. Javascript has confused them. Tell me where is the lexer in Javascript? or the virtual machine? The point I'm making about LISP is that in LISP you don't define a function, it IS the function. Instead of f(x,y) = x+y, which is how you code a traditional program, you only show/program f itself. In the lambda calculus, it is expressed f = /(x,y).x+y where / is the lambda character, not f(x) or f OF x. Do you see why the difference is subtle? Java has a virtual machine. Python has a virtual machine. Do you think Chrome and these browsers include virtual machines to run on 50 different architectures? I'm actually curious if anyone has a non-magical answer to it. We all assume that computers can only act logically and have always acted such. But they way some people act and the way somethings are being done is making questions. Perhaps I'm the only one. As for chrome and its specific VM http://en.wikipedia.org/wiki/V8_%28JavaScript_engine%29 As for the rest... We are as confused by your confusions as you are [Your continual assertions about everyone everywhere being totally confused reminds me of my first trip to N America where I was unnerved to find everyone driving on the wrong side of the road] Thanks for that note, actually, it's quite apropos. I'm a space cowboy who's trying to tell people what I've seen from outside the fishbowl. To those in the fishbowl, they don't think that they're in water, because they've never been outside of it--the concept doesn't exist. But I left the orthodoxy (without throwing it away) and saw from the outside. Zen is one of the nodal points of that outside. And from that view, I can tell you that Python has abandoned much of it`s Zen to the point that is now only held by the programmers who still believe in it like some lost ancient tablet that they keep in their pocket. It can come back easily, though. One conversation in person with a whiteboard would do it. I'm starting to see why I'm seen as such an iconoclast: I've been outside of it for a long time and have taken it for granted -- I couldn't see my own water. Oh well. Maybe the relationship will be mended with the community, or maybe not. My psychological states go from one extreme to another, so that's my thing in case anyone was wondering what my issue is--it's happened ever since getting tazered. It sucked. Mark -- https://mail.python.org/mailman/listinfo/python-list
Re: pip grabs tar.gz file instead of whl?
On Sat, May 16, 2015 at 11:58 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The way I see it, pip is great for handling the most common case where you just want to name a package and say go fetch, but if you want to override its decisions, you should use the lower-level facilities eg manual downloading and setup.py. It's like with Debian packages: I can type sudo apt-get install blah and it'll run off and grab it, check its signatures, make sure everything's right, and then install it; but if I want to install something from a different location, the best way is usually to download it manually, do my own checking, and then sudo dpkg -i blah.deb to actually install it - no apt-get involvement at all. This shouldn't normally be a problem; you don't *have* to use pip here, you just want to end up with the package properly installed. ChrisA Being on Windows, as I said at the beginning of the thread, the biggest problem is that setup.py can't find VS if there is no whl file to install. Hence it is far easier to get the binaries from elsewhere. Hopefully this problem will disappear in the future as the whl standard becomes prevelant. As for sudo I always thought that was a type of Japanese wrestling :) That'll be sumo :) sudo is su do this, and it's like using su, then doing something, and then dropping out again. (su gives you a new prompt as superuser, or as some other user.) I don't know what the exact installation steps are for a whl, which is why I mentioned setup.py. Whatever those lower-level facilities are, those are what you'd use once you decide to skip pip and do your own downloading. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Minimising stack trace
On Saturday, May 16, 2015 at 6:54:23 AM UTC+5:30, Ned Batchelder wrote: On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote: On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote: Cecil Westerhof wrote: While playing with recursion I get: RuntimeError: maximum recursion depth exceeded in comparison But then I get a very long stack trace. Is there a way to make this a lot shorter. Now I 'lose' interesting information because of the length of the stack trace. http://www.catb.org/~esr/faqs/smart-questions.html There ought to be a website that explains how to give smart answers. Or one that explains how to be kind to those that don't know the ropes yet. Though pointing impatient experts at a website probably won't get them to change any more than brusquely pointing newbs at a website will. How about a 'answerer-quota' ? « Once for every 20 useful answers you are allowed to vent on any subject of your choice... - Stupid questions (and point to catb's smart questions) - Google groups - Top posting - Html mail » -- https://mail.python.org/mailman/listinfo/python-list
