Re: [Python-Dev] Free lists
Here is comparable statistic collected from tests ran with an executable buil with COUNT_ALLOCS. typecount % acc.% tuple 448855278 29.50% 29.50% frame 203515969 13.38% 42.88% str 182658237 12.01% 54.89% builtin_function_or_method 156724634 10.30% 65.19% int 106561963 7.00% 72.19% method 88269762 5.80% 78.00% list 50340630 3.31% 81.31% slice36650028 2.41% 83.71% dict 34429310 2.26% 85.98% generator33035375 2.17% 88.15% bytes29230573 1.92% 90.07% function 24953392 1.64% 91.71% list_iterator21236155 1.40% 93.11% tuple_iterator 16800947 1.10% 94.21% cell 16369317 1.08% 95.29% float 7079162 0.47% 95.75% _sre.SRE_Match6342612 0.42% 96.17% set 5322829 0.35% 96.52% TokenInfo 5077251 0.33% 96.85% code 3643664 0.24% 97.09% traceback 3510709 0.23% 97.32% memoryview2860799 0.19% 97.51% managedbuffer 2762975 0.18% 97.69% method-wrapper2590642 0.17% 97.86% Name 1681233 0.11% 97.97% bytearray 1598429 0.11% 98.08% _io.StringIO 1439456 0.09% 98.17% weakref 1341485 0.09% 98.26% super 911811 0.06% 98.32% range 798254 0.05% 98.37% ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode literals in Python 2.7
Glenn Linderman wrote: > Is this going to get released in 3.5, I hope? Python 3 is pretty > limited without some solution for Unicode on the console... probably the > biggest deficiency I have found in Python 3, since its introduction. It > has great Unicode support for files and processing, which convinced me > to switch from Perl, and I like so much else about it, that I can hardly > code in Perl any more (I still support a few Perl programs, but have > ported most of them to Python). I'd love to see it included in 3.5, but I doubt that will happen. For one thing, it's only two weeks till beta 1, which is feature freeze. And mainly, my package is mostly hacking into existing Python environment. A proper implementation would need some changes in Python someone would have to do. See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm not competent to write a patch myself and I have also no feedback to the proposed idea. On the other hand, using the package is good enough for me so I didn't further bring attention to the proposal. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Mac popups running make test
I haven't run the test suite in awhile. I am in the midst of running it on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: I assume this is testing some server listening on localhost. Is this a new thing, either with the Python test suite or with Mac OS X? (I'd normally be hidden behind a NAT firewall, but at the moment I am on a miserable public connection in a Peet's Coffee, so it takes on slightly more importance...) I've also seen the Crash Reporter pop up many times, but as far as I could tell, in all cases the test suite output told me it was expected. Perhaps tests which listen for network connections should also mention that, at least on Macs? Thx, Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mac popups running make test
On Sun, May 10, 2015 at 10:04 AM Skip Montanaro wrote: > I haven't run the test suite in awhile. I am in the midst of running it on > my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: > > > > I assume this is testing some server listening on localhost. Is this a new > thing, either with the Python test suite or with Mac OS X? (I'd normally be > hidden behind a NAT firewall, but at the moment I am on a miserable public > connection in a Peet's Coffee, so it takes on slightly more importance...) > It's not new. > > I've also seen the Crash Reporter pop up many times, but as far as I could > tell, in all cases the test suite output told me it was expected. Perhaps > tests which listen for network connections should also mention that, at > least on Macs? > Wouldn't hurt. Just requires tracking down which test(s) triggers it (might be more than one and I don't know if answering that popup applies for the rest of the test execution or once per test if you use -j). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Free lists
On 05/09/2015 11:22 PM, Serhiy Storchaka wrote: On 10.05.15 02:25, Ian Cordasco wrote: Can you share how you gathered them so someone could run them on a 64-bit build? This is quick and dirty patch. It generates 8 GB log file! I ran it under 64-bit Linux. Actually it generated a 10GB log file. It was stalled at test_multiprocessing_fork for five hours so I killed it. What follows are the (apparently) partial results. I think it makes a good case for a one-element freelist for 64-bit builds. //arry/ -- typecount % acc.% builtin_function_or_method 131028598 37.25% 37.25% method 52062496 14.80% 52.05% int 47600237 13.53% 65.59% str 43841584 12.46% 78.05% generator14038624 3.99% 82.04% float 8617481 2.45% 84.49% list_iterator 8214121 2.34% 86.83% bytes 7884898 2.24% 89.07% tuple_iterator5172174 1.47% 90.54% _io.StringIO 3482733 0.99% 91.53% set 3335168 0.95% 92.48% str_iterator 2856373 0.81% 93.29% list 2245981 0.64% 93.93% dict 1682253 0.48% 94.41% method-wrapper1574412 0.45% 94.86% function 1475393 0.42% 95.28% traceback 1417094 0.40% 95.68% tuple 1181899 0.34% 96.01% memoryview1103226 0.31% 96.33% cell 1047245 0.30% 96.63% managedbuffer 1044764 0.30% 96.92% bytearray 714337 0.20% 97.13% range_iterator 498240 0.14% 97.27% range 485325 0.14% 97.41% super 473542 0.13% 97.54% map446608 0.13% 97.67% frame 426570 0.12% 97.79% set_iterator 424526 0.12% 97.91% Leaf 391824 0.11% 98.02% symtable 376815 0.11% 98.13% int 47600237 13.53% 0 294964 0.62% 0.62% 1 36135772 75.92% 76.53% 24504046 9.46% 86.00% 32109837 4.43% 90.43% 41277995 2.68% 93.11% 5 542775 1.14% 94.25% 6 485451 1.02% 95.27% ... bytes 7884898 2.24% 0849 0.01% 0.01% 1 250357 3.18% 3.19% 2 450310 5.71% 8.90% 3 259659 3.29% 12.19% 41157554 14.68% 26.87% 5 77493 0.98% 27.85% 6 139816 1.77% 29.63% 7 165399 2.10% 31.72% 8 191821 2.43% 34.16% 9 63009 0.80% 34.96% 10 48751 0.62% 35.57% 11 50505 0.64% 36.22% 12 94186 1.19% 37.41% 13 33927 0.43% 37.84% 14 123546 1.57% 39.41% 15 36565 0.46% 39.87% 16 447183 5.67% 45.54% 17 186609 2.37% 47.91% 181301737 16.51% 64.42% ... tuple 1181899 0.34% 0 47 0.00% 0.00% 1 120156 10.17% 10.17% 2 340983 28.85% 39.02% 3 80924 6.85% 45.87% 4 78908 6.68% 52.54% 5 35502 3.00% 55.55% 6 171292 14.49% 70.04% 7 136474 11.55% 81.59% 8 48435 4.10% 85.69% ... ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mac popups running make test
On Sun, May 10, 2015 at 5:07 PM, Brett Cannon wrote: > > > On Sun, May 10, 2015 at 10:04 AM Skip Montanaro > wrote: > >> I haven't run the test suite in awhile. I am in the midst of running it >> on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: >> >> >> >> I assume this is testing some server listening on localhost. Is this a >> new thing, either with the Python test suite or with Mac OS X? (I'd >> normally be hidden behind a NAT firewall, but at the moment I am on a >> miserable public connection in a Peet's Coffee, so it takes on slightly >> more importance...) >> > > It's not new. > Indeed, I've run into this as well. > >> I've also seen the Crash Reporter pop up many times, but as far as I >> could tell, in all cases the test suite output told me it was expected. >> Perhaps tests which listen for network connections should also mention >> that, at least on Macs? >> > > Wouldn't hurt. Just requires tracking down which test(s) triggers it > (might be more than one and I don't know if answering that popup applies > for the rest of the test execution or once per test if you use -j). > If anyone starts working on this, let me know if I can help, e.g. trying things on my own Mac. - Tal ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mac popups running make test
On Sun, May 10, 2015 at 9:07 PM, Carol Willing < [email protected]> wrote: > > On 5/10/15 10:29 AM, Tal Einat wrote: > > On Sun, May 10, 2015 at 5:07 PM, Brett Cannon wrote: > >> >> >> On Sun, May 10, 2015 at 10:04 AM Skip Montanaro >> wrote: >> >>> I haven't run the test suite in awhile. I am in the midst of running it >>> on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup: >>> >>> >>> >>> I assume this is testing some server listening on localhost. Is this a >>> new thing, either with the Python test suite or with Mac OS X? (I'd >>> normally be hidden behind a NAT firewall, but at the moment I am on a >>> miserable public connection in a Peet's Coffee, so it takes on slightly >>> more importance...) >>> >> >> It's not new. >> > > Indeed, I've run into this as well. > > >> >>> I've also seen the Crash Reporter pop up many times, but as far as I >>> could tell, in all cases the test suite output told me it was expected. >>> Perhaps tests which listen for network connections should also mention >>> that, at least on Macs? >>> >> >> Wouldn't hurt. Just requires tracking down which test(s) triggers it >> (might be more than one and I don't know if answering that popup applies >> for the rest of the test execution or once per test if you use -j). >> > > If anyone starts working on this, let me know if I can help, e.g. trying > things on my own Mac. > >I believe that the message has to do with OS X's sandboxing > implementation and the setting of the sandbox's entitlement keys. Here's an > Apple doc: > https://developer.apple.com/library/ios/documentation/Miscellaneous/Reference/EntitlementKeyReference/Chapters/EnablingAppSandbox.html#//apple_ref/doc/uid/TP40011195-CH4-SW9 > > I'm unaware of a way to work around this other than using Apple's code > signing or adjusting target build settings in XCode :( If anyone knows a > good way to workaround or manually set permission (other than clicking the > Allow button), I would be interested. > I was reading about this a few weeks ago an recall finding a way to ad-hoc sign the built python executable. Here's a link below. I haven't tried this, though, and don't know if it would work with a python executable rather than a proper OSX app. If it does work, it would be useful to add this as a tool and/or mention it in the developer docs. http://apple.stackexchange.com/a/121010 - Tal Einat ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is it kosher to use a buffer after release?
In Python's argument parsing code (convertsimple in Python/getargs.c), a
couple of format units* accept "read-only bytes-like objects", aka
read-only buffer objects. They call a helper function called
convertbuffer() which uses the buffer protocol to extract a pointer to
the memory.
Here's the relevant bit of code:
static Py_ssize_t
convertbuffer(PyObject *arg, void **p, char **errmsg)
{
Py_buffer view;
...
if (getbuffer(arg, &view, errmsg) < 0)
return -1;
count = view.len;
*p = view.buf;
PyBuffer_Release(&view);
return count;
}
getbuffer() uses the buffer protocol to fill in the "view" buffer. If
it's successful, "view" is a valid buffer. We store the pointer to the
buffer's memory in output parameter p.
THEN WE RELEASE THE BUFFER.
THEN WE RETURN TO THE CALLER.
In case you missed the big helpful capital letters, we are returning a
pointer given to us by PyObject_GetBuffer(), which we have already
released by calling PyBuffer_Release(). The buffer protocol
documentation for bf_releasebuffer makes it sound like this pointer
could easily be invalid after the release call finishes.
Am I missing something, or is this code relying on an implementation
detail it shouldn't--namely that you can continue using a pointer to
some (most? all?) buffer memory even after releasing it?
//arry/
* Specifically: s# y y# z#
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is it kosher to use a buffer after release?
On 10.05.15 21:28, Larry Hastings wrote:
In Python's argument parsing code (convertsimple in Python/getargs.c), a
couple of format units* accept "read-only bytes-like objects", aka
read-only buffer objects. They call a helper function called
convertbuffer() which uses the buffer protocol to extract a pointer to
the memory.
Here's the relevant bit of code:
static Py_ssize_t
convertbuffer(PyObject *arg, void **p, char **errmsg)
{
Py_buffer view;
...
if (getbuffer(arg, &view, errmsg) < 0)
return -1;
count = view.len;
*p = view.buf;
PyBuffer_Release(&view);
return count;
}
getbuffer() uses the buffer protocol to fill in the "view" buffer. If
it's successful, "view" is a valid buffer. We store the pointer to the
buffer's memory in output parameter p.
THEN WE RELEASE THE BUFFER.
THEN WE RETURN TO THE CALLER.
In case you missed the big helpful capital letters, we are returning a
pointer given to us by PyObject_GetBuffer(), which we have already
released by calling PyBuffer_Release(). The buffer protocol
documentation for bf_releasebuffer makes it sound like this pointer
could easily be invalid after the release call finishes.
Am I missing something, or is this code relying on an implementation
detail it shouldn't--namely that you can continue using a pointer to
some (most? all?) buffer memory even after releasing it?
You are missing following code:
if (pb != NULL && pb->bf_releasebuffer != NULL) {
*errmsg = "read-only bytes-like object";
return -1;
}
convertbuffer() is applicable only for types for which
PyBuffer_Release() is no-op. That is why there are different format
units for read-only buffers and for general buffers. That is why new
buffer protocol was introduced.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
