[Python-Dev] Release cycle question
Hello everyone. I'm writing to ask if Python uses a "fixed" release time or if it depends strongly on something else. In example, Blender does and since I'm diving into Python because I would like to extend it, I would like to know if my work will have a default lifetime or not. By the way, Python 3 changed the game AFAIK, will another major change come short? Thanks in advance for any help. Bye, Ivano. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release cycle question
On Mon, Jan 30, 2012 at 7:51 PM, Ivano wrote: > Hello everyone. > I'm writing to ask if Python uses a "fixed" release > time or if it depends strongly on something else. > In example, Blender does and since I'm diving > into Python because I would like to extend it, I > would like to know if my work will have a default > lifetime or not. Hi Ivano, The current release cycle is documented in the developer's guide: http://docs.python.org/devguide/devcycle.html At this point in time, there are two official python.org releases: Python 2.7 and 3.2 2.7 was released in July 2010 and will receive maintenance updates until around July 2015 (as it is the final release in the 2.x series) 3.2 was released in February 2011 and will receive maintenance updates until around August this year (but will receive further source-only security updates until around February 2016) 3.3 is due for release in August this year. However, those are the official support dates specifically for python-dev. OS vendors such as Red Hat and Canonical provide support for older versions of Python as part of their enterprise releases (e.g. RHEL5 is still supported by Red Hat and ships with Python 2.4, even though python-dev ended upstream security updates for 2.4 in 2009) > By the way, Python 3 changed the game AFAIK, > will another major change come short? No, as noted on the development cycle page, changes on the scale of those between Python 2 and Python 3 are not expected any time in the near future. I'd personally be surprised if anything like that transition happened again within the next decade. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
On Fri, Jan 27, 2012 at 21:33, Benjamin Peterson wrote: > 2012/1/27 Steven D'Aprano : > > Benjamin Peterson wrote: > >> > >> Hello everyone, > >> In effort to get a fix out before Perl 6 goes mainstream, Barry and I > >> have decided to pronounce on what we want for our stable releases. > >> What we have decided is that > >> 1. Simple hash randomization is the way to go. We think this has the > >> best chance of actually fixing the problem while being fairly > >> straightforward such that we're comfortable putting it in a stable > >> release. > >> 2. It will be off by default in stable releases and enabled by an > >> envar at runtime. This will prevent code breakage from dictionary > >> order changing as well as people depending on the hash stability. > > > Great! > > > > Do you have the expectation that it will become on by default in some > future > > release? > > Yes, 3.3. The solution in 3.3 could even be one of the more > sophisticated proposals we have today. I think that would be good. And I would even argue we remove support for turning it off to force people to no longer lean on dict ordering as a crutch (in 3.3 obviously). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
On Jan 30, 2012, at 12:03 PM, Brett Cannon wrote: >I think that would be good. And I would even argue we remove support for >turning it off to force people to no longer lean on dict ordering as a >crutch (in 3.3 obviously). Yes, please! -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 1/29/2012 4:39 PM, Gregory P. Smith wrote: > An example of this working: ipaddr is ready to go in. It got the > eyeballs and API modifications while still a pypi library as a result > of the discussion around the time it was originally suggested as being > added. I or any other committers have simply not added it yet. This is wrong. PEP 3144 was not pronounced upon, so ipaddr is not just waiting for someone to commit it; it's waiting on consensus and pronouncement. PEP 3144 wasn't pronounced upon because there were significant disagreements about the design of the API proposed in the PEP. As it stands, I believe the authorship of ipaddr either decided that they were not going to compromise their module or lost interest. See Nick Coghlan's summary: http://mail.python.org/pipermail//python-ideas/2011-August/011305.html -- Scott Dial [email protected] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Maybe that's another example of waiting too long for the perfect decision though. In the last ~12 months, ipaddr was downloaded at least 11,000 times from its home (http://code.google.com/p/ipaddr-py/downloads/list). There's been a fair amount of changes over that time and a new release was put out 10 days ago. What are the stats for the "competing" package? --Guido On Mon, Jan 30, 2012 at 10:19 AM, Scott Dial wrote: > On 1/29/2012 4:39 PM, Gregory P. Smith wrote: >> An example of this working: ipaddr is ready to go in. It got the >> eyeballs and API modifications while still a pypi library as a result >> of the discussion around the time it was originally suggested as being >> added. I or any other committers have simply not added it yet. > > This is wrong. PEP 3144 was not pronounced upon, so ipaddr is not just > waiting for someone to commit it; it's waiting on consensus and > pronouncement. > > PEP 3144 wasn't pronounced upon because there were significant > disagreements about the design of the API proposed in the PEP. As it > stands, I believe the authorship of ipaddr either decided that they were > not going to compromise their module or lost interest. > > See Nick Coghlan's summary: > > http://mail.python.org/pipermail//python-ideas/2011-August/011305.html > > -- > Scott Dial > [email protected] > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
On 2012-01-30 01:46, Victor Stinner wrote: But why would you want to pass a float? It seems like API abuse to me. If something should be changed, Semaphore(arg) should raise a TypeError if arg is not an integer. Short version: I propose the the change to be -while self._value == 0: +while self._value < 1: This should not change the flow when Semaphore._value is an int. Longer explanation: I thought it is surprising to use math.floor() for threading.Semaphore, but now as you propose, we will need to use something like int(math.floor(value)) in Python2.x - which is even more surprising. That is because math.floor() (and round() for that matter) return a float object in Python2.x. Note: isinstance(4.0, numbers.Integral) is False, even in Python3.x, but until now 4.0 was valid as a value for Semaphore(). Also, using the builtin int()/math.trunc() on a float is probably not what you want here, but rather math.floor(). The value argument given to threading.Semaphore() is really a duck (or an object) that can be compared to 0 and 1, incremented by 1 and decremented by 1. These are properties that fit float. Why should you force the entire builtin int behavior on that object? I agree that using a float as the counter smells bad, but at times you might have something like a fractional resource (which is different from a floating point number). In such cases Semaphore.acquire(), after the tiny patch above, can be thought as checking if you have at least one "unit of resource" available. If you do have at least one such resource - acquire it. This will make sure the invariant "The counter can never go below zero" holds. Regards, TB ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
TB, what's your use case for passing a float to a semaphore? Semaphores are conceptually tied to integers. You've kept arguing a few times now that the workaround you need are clumsy, but you've not explained why you're passing floats in the first place. A "fractional resource" just doesn't sound like a real use case to me. On Mon, Jan 30, 2012 at 10:40 AM, T.B. wrote: > > On 2012-01-30 01:46, Victor Stinner wrote: >>> >>> >>> But why would you want to pass a float? It seems like API abuse to me. >> >> >> If something should be changed, Semaphore(arg) should raise a >> TypeError if arg is not an integer. >> > Short version: > I propose the the change to be > > - while self._value == 0: > + while self._value < 1: > This should not change the flow when Semaphore._value is an int. > > Longer explanation: > I thought it is surprising to use math.floor() for threading.Semaphore, but > now as you propose, we will need to use something like > int(math.floor(value)) in Python2.x - which is even more surprising. That is > because math.floor() (and round() for that matter) return a float object in > Python2.x. > > Note: isinstance(4.0, numbers.Integral) is False, even in Python3.x, but > until now 4.0 was valid as a value for Semaphore(). Also, using the builtin > int()/math.trunc() on a float is probably not what you want here, but rather > math.floor(). > > The value argument given to threading.Semaphore() is really a duck (or an > object) that can be compared to 0 and 1, incremented by 1 and decremented by > 1. These are properties that fit float. Why should you force the entire > builtin int behavior on that object? > > I agree that using a float as the counter smells bad, but at times you might > have something like a fractional resource (which is different from a > floating point number). In such cases Semaphore.acquire(), after the tiny > patch above, can be thought as checking if you have at least one "unit of > resource" available. If you do have at least one such resource - acquire it. > This will make sure the invariant "The counter can never go below zero" > holds. > > Regards, > TB > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sun, 29 Jan 2012 16:42:28 +1000 Nick Coghlan wrote: > On Sun, Jan 29, 2012 at 1:29 PM, Guido van Rossum wrote: > > On Sat, Jan 28, 2012 at 5:33 PM, Nick Coghlan wrote: > >> I'm willing to go along with that (especially given your report of > >> AppEngine's experience with the "labs" namespace). > >> > >> Can we class this as a pronouncement on PEP 408? That is, "No to > >> adding a __preview__ namespace, but yes to adding regex directly for > >> 3.3"? > > > > Yup. We seem to have a tendency to over-analyze decisions a bit lately > > (witness the hand-wringing about the hash collision DoS attack). > > I have now updated PEP 408 accordingly (i.e. rejected, but with a > specific note about regex). It would be nice if that pronouncement or decision could outline the steps required to include an "experimental" module in the stdlib, and the steps required to move it from "experimental" to "stable". Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
Hello, > Could you try benchmarking with the "standard" benchmarks: > http://hg.python.org/benchmarks/ > and see what sort of performance gains you get? > Yeah, of course. I already did. Refere to the page listed below for details. I did not look into the results yet, though. > How portable is the threaded interpreter? > Well, you can implement threaded code on any machine that support indirect branch instructions. Fortunately, GCC supports the "label-as-values" feature, which makes it available on any machine that supports GCC. My optimizations themselves are portable, and I tested them on a PowerPC for my thesis, too. (AFAIR, llvm supports this feature, too.) > Do you have a public repository for the code, so we can take a look? > I have created a patch (as Benjamin wanted) and put all of the resources (i.e., benchmark results and the patch itself) on my home page: http://www.ics.uci.edu/~sbruntha/pydev.html Regards, --stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
Hello, > Well, you can implement threaded code on any machine that support > indirect branch instructions. Fortunately, GCC supports the > "label-as-values" feature, which makes it available on any machine > that supports GCC. My optimizations themselves are portable, and I > tested them on a PowerPC for my thesis, too. (AFAIR, llvm supports > this feature, too.) Well, you're aware that Python already uses threaded code where available? Or are you testing against Python 2? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
> Well, you're aware that Python already uses threaded code where > available? Or are you testing against Python 2? > Yes, and I am building on that. --stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
On Tue, Jan 31, 2012 at 3:03 AM, Brett Cannon wrote: > I think that would be good. And I would even argue we remove support for > turning it off to force people to no longer lean on dict ordering as a > crutch (in 3.3 obviously). On-by-default should be enough to cover that. Just as we allow people to force the random seed to reproduce particular sequences, there's value in being able to increase determinism in cases where the collision attack isn't a concern. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #8828: Add new function os.replace(), for cross-platform renaming with
Should this end up being used in importlib through _os?
On Mon, Jan 30, 2012 at 16:11, antoine.pitrou wrote:
> http://hg.python.org/cpython/rev/80ddbd87
> changeset: 74689:80ddbd87
> user:Antoine Pitrou
> date:Mon Jan 30 22:08:52 2012 +0100
> summary:
> Issue #8828: Add new function os.replace(), for cross-platform renaming
> with overwriting.
>
> files:
> Doc/library/os.rst| 18 +-
> Lib/test/test_os.py | 12 ++
> Misc/NEWS | 3 +
> Modules/posixmodule.c | 55 +-
> 4 files changed, 69 insertions(+), 19 deletions(-)
>
>
> diff --git a/Doc/library/os.rst b/Doc/library/os.rst
> --- a/Doc/library/os.rst
> +++ b/Doc/library/os.rst
> @@ -1889,8 +1889,9 @@
>Unix flavors if *src* and *dst* are on different filesystems. If
> successful,
>the renaming will be an atomic operation (this is a POSIX requirement).
> On
>Windows, if *dst* already exists, :exc:`OSError` will be raised even if
> it is a
> - file; there may be no way to implement an atomic rename when *dst*
> names an
> - existing file.
> + file.
> +
> + If you want cross-platform overwriting of the destination, use
> :func:`replace`.
>
>Availability: Unix, Windows.
>
> @@ -1908,6 +1909,19 @@
> permissions needed to remove the leaf directory or file.
>
>
> +.. function:: replace(src, dst)
> +
> + Rename the file or directory *src* to *dst*. If *dst* is a directory,
> + :exc:`OSError` will be raised. If *dst* exists and is a file, it will
> + be replaced silently if the user has permission. The operation may
> fail
> + if *src* and *dst* are on different filesystems. If successful,
> + the renaming will be an atomic operation (this is a POSIX requirement).
> +
> + Availability: Unix, Windows
> +
> + .. versionadded:: 3.3
> +
> +
> .. function:: rmdir(path)
>
>Remove (delete) the directory *path*. Only works when the directory is
> diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
> --- a/Lib/test/test_os.py
> +++ b/Lib/test/test_os.py
> @@ -129,6 +129,18 @@
> self.fdopen_helper('r')
> self.fdopen_helper('r', 100)
>
> +def test_replace(self):
> +TESTFN2 = support.TESTFN + ".2"
> +with open(support.TESTFN, 'w') as f:
> +f.write("1")
> +with open(TESTFN2, 'w') as f:
> +f.write("2")
> +self.addCleanup(os.unlink, TESTFN2)
> +os.replace(support.TESTFN, TESTFN2)
> +self.assertRaises(FileNotFoundError, os.stat, support.TESTFN)
> +with open(TESTFN2, 'r') as f:
> +self.assertEqual(f.read(), "1")
> +
>
> # Test attributes on return values from os.*stat* family.
> class StatAttributeTests(unittest.TestCase):
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -463,6 +463,9 @@
> Library
> ---
>
> +- Issue #8828: Add new function os.replace(), for cross-platform renaming
> + with overwriting.
> +
> - Issue #13848: open() and the FileIO constructor now check for NUL
> characters in the file name. Patch by Hynek Schlawack.
>
> diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
> --- a/Modules/posixmodule.c
> +++ b/Modules/posixmodule.c
> @@ -3280,17 +3280,16 @@
> #endif /* HAVE_SETPRIORITY */
>
>
> -PyDoc_STRVAR(posix_rename__doc__,
> -"rename(old, new)\n\n\
> -Rename a file or directory.");
> -
> -static PyObject *
> -posix_rename(PyObject *self, PyObject *args)
> +static PyObject *
> +internal_rename(PyObject *self, PyObject *args, int is_replace)
> {
> #ifdef MS_WINDOWS
> PyObject *src, *dst;
> BOOL result;
> -if (PyArg_ParseTuple(args, "UU:rename", &src, &dst))
> +int flags = is_replace ? MOVEFILE_REPLACE_EXISTING : 0;
> +if (PyArg_ParseTuple(args,
> + is_replace ? "UU:replace" : "UU:rename",
> + &src, &dst))
> {
> wchar_t *wsrc, *wdst;
>
> @@ -3301,16 +3300,17 @@
> if (wdst == NULL)
> return NULL;
> Py_BEGIN_ALLOW_THREADS
> -result = MoveFileW(wsrc, wdst);
> +result = MoveFileExW(wsrc, wdst, flags);
> Py_END_ALLOW_THREADS
> if (!result)
> -return win32_error("rename", NULL);
> +return win32_error(is_replace ? "replace" : "rename", NULL);
> Py_INCREF(Py_None);
> return Py_None;
> }
> else {
> PyErr_Clear();
> -if (!PyArg_ParseTuple(args, "O&O&:rename",
> +if (!PyArg_ParseTuple(args,
> + is_replace ? "O&O&:replace" : "O&O&:rename",
> PyUnicode_FSConverter, &src,
> PyUnicode_FSConverter, &dst))
> return NULL;
> @@ -3319,15 +3319,15 @@
> goto error;
>
> Py_BEGIN_ALLOW_THREADS
> -result = MoveFileA(PyBytes_AS_STRING(src),
> - PyBytes_AS_STRING(dst));
> +result = MoveFileExA(PyBytes_AS_STRING(src
Re: [Python-Dev] cpython: Issue #8828: Add new function os.replace(), for cross-platform renaming with
On Mon, 30 Jan 2012 16:26:30 -0500 Brett Cannon wrote: > Should this end up being used in importlib through _os? Yes, probably. I hadn't thought about that. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 3144 ipaddr module (was Re: PEP 408 -- Standard library __preview__ package)
On Tue, Jan 31, 2012 at 4:19 AM, Scott Dial wrote: > PEP 3144 wasn't pronounced upon because there were significant > disagreements about the design of the API proposed in the PEP. As it > stands, I believe the authorship of ipaddr either decided that they were > not going to compromise their module or lost interest. > > See Nick Coghlan's summary: > > http://mail.python.org/pipermail//python-ideas/2011-August/011305.html Peter Moody actually addressed all my comments from last year (alas, I forgot that python-ideas got dropped from the latter part of the email chain, so it became a private discussion between Peter, Guido and myself). I apparently got distracted by other issues and never followed up on Peter's final review request. The branch with the relevant changes is here (these weren't added back into ipaddr mainline since they aren't all backwards compatible with the existing ipaddr API): http://code.google.com/p/ipaddr-py/source/browse/#svn%2Fbranches%2F3144 Peter was very responsive and accommodating during that discussion :) (The notes below are an edited version of Peter's off-list reply to me from last year, reflecting the final state of the ipaddr 3144 branch) On Mon, Aug 29, 2011 at 7:09 PM, Nick Coghlan wrote: I believe the PEP would be significantly more palatable with the following changes/additions: 1. Draft ReStructuredText documentation for inclusion in the stdlib docs (still needed) 2. Removal of the "ip" attribute of IP network objects (since it makes the nominal "networks" behave like IP interface definitions) the Class hierarchy now looks like: _IPAddrBase(object) # mother of everything _BaseAddress(_IPAddrBase) # base for addresses _ BaseNetwork(_IPAddrBase) # base for networks and interfaces, could use be renamed. _BaseV4(object) # ipv4 base _BaseV6(object) # ipv6 base IPv4Address(_BaseV4, _BaseAddress) IPv4Interface(_BaseV4, _BaseNetwork) IPv4Network(IPv4Interface) IPv6Address(_BaseV6, _BaseAddress) IPv6Interface(_BaseV6, _BaseNetwork) IPv6Network(IPv6Interface) (essentially, the current ipaddr "Network" objects become "Interface" objects in PEP 3144, with a new strict "Network" object that has no ip attribute) 3. "network" property renamed to "netaddr" (since it returns an address object rather than a network object) renamed to network_address. did the same for the broadcast_address. 4. "strict" parameter removed from class signatures, replaced with class method for non-strict behaviour 'strict' is gone, just create IPv*Interface objects or use the ip_interface API instead. Network objects are always strict. 5. Factory functions renamed so they don't look like class names (ip_network, ip_address, ip) Now ip_address, ip_network, ip_interface 6. "strict" parameter on factory functions modified to default to True rather than False 'strict' is gone. Interfaces allow a host IP, Networks don't. 7. Addition of an explicit "IPInterface" class to cover the association of an address with a specific network that is currently handled by storing arbitrary addresses on IP network objects done. So with a cleanup of the docstrings (and creation of some ReST docs based on them) a definite +1 from me for inclusion of ipaddr (based on the 3144 branch in SVN) in 3.3. (with the tweaks to the API, we may want to use a different name like "ipaddress" or "iptools", though - otherwise people could be legitimately confused by the differences relative to the PyPI "ipaddr" module) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 ipaddr module (was Re: PEP 408 -- Standard library __preview__ package)
On Mon, Jan 30, 2012 at 1:44 PM, Nick Coghlan wrote: > On Tue, Jan 31, 2012 at 4:19 AM, Scott Dial > wrote: >> PEP 3144 wasn't pronounced upon because there were significant >> disagreements about the design of the API proposed in the PEP. As it >> stands, I believe the authorship of ipaddr either decided that they were >> not going to compromise their module or lost interest. >> >> See Nick Coghlan's summary: >> >> http://mail.python.org/pipermail//python-ideas/2011-August/011305.html > > Peter Moody actually addressed all my comments from last year (alas, I > forgot that python-ideas got dropped from the latter part of the email > chain, so it became a private discussion between Peter, Guido and > myself). I apparently got distracted by other issues and never > followed up on Peter's final review request. The branch with the > relevant changes is here (these weren't added back into ipaddr > mainline since they aren't all backwards compatible with the existing > ipaddr API): > http://code.google.com/p/ipaddr-py/source/browse/#svn%2Fbranches%2F3144 > > Peter was very responsive and accommodating during that discussion :) > > (The notes below are an edited version of Peter's off-list reply to me > from last year, reflecting the final state of the ipaddr 3144 branch) > > On Mon, Aug 29, 2011 at 7:09 PM, Nick Coghlan wrote: > > I believe the PEP would be significantly more palatable with the > following changes/additions: > 1. Draft ReStructuredText documentation for inclusion in the stdlib docs > > (still needed) > > 2. Removal of the "ip" attribute of IP network objects (since it makes > the nominal "networks" behave like IP interface definitions) > > the Class hierarchy now looks like: > > _IPAddrBase(object) # mother of everything > _BaseAddress(_IPAddrBase) # base for addresses > _ BaseNetwork(_IPAddrBase) # base for networks and interfaces, could > use be renamed. > _BaseV4(object) # ipv4 base > _BaseV6(object) # ipv6 base > > IPv4Address(_BaseV4, _BaseAddress) > IPv4Interface(_BaseV4, _BaseNetwork) > IPv4Network(IPv4Interface) > > IPv6Address(_BaseV6, _BaseAddress) > IPv6Interface(_BaseV6, _BaseNetwork) > IPv6Network(IPv6Interface) > > (essentially, the current ipaddr "Network" objects become "Interface" > objects in PEP 3144, with a new strict "Network" object that has no ip > attribute) > > 3. "network" property renamed to "netaddr" (since it returns an > address object rather than a network object) > > renamed to network_address. > did the same for the broadcast_address. > > 4. "strict" parameter removed from class signatures, replaced with > class method for non-strict behaviour > > 'strict' is gone, just create IPv*Interface objects or use the > ip_interface API instead. Network objects are always strict. > > > 5. Factory functions renamed so they don't look like class names > (ip_network, ip_address, ip) > > Now ip_address, ip_network, ip_interface > > > 6. "strict" parameter on factory functions modified to default to True > rather than False > > 'strict' is gone. Interfaces allow a host IP, Networks don't. > > 7. Addition of an explicit "IPInterface" class to cover the > association of an address with a specific network that is currently > handled by storing arbitrary addresses on IP network objects > > done. > > So with a cleanup of the docstrings (and creation of some ReST docs > based on them) a definite +1 from me for inclusion of ipaddr (based on > the 3144 branch in SVN) in 3.3. (with the tweaks to the API, we may > want to use a different name like "ipaddress" or "iptools", though - > otherwise people could be legitimately confused by the differences > relative to the PyPI "ipaddr" module) Cleaning up the docstrings and re-tooling the PEP was where I stalled after addressing your comments. Easy enough to complete if there's still interest. Note, http://pypi.python.org/pypi/ipaddr is actually the same module, but down a few versions. I'm not sure if your concern is about the same library having such a different api or if you had thought they were completely different libraries. Cheers, peter > Cheers, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia -- Peter Moody Google 1.650.253.7306 Security Engineer pgp:0xC3410038 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Tue, Jan 31, 2012 at 4:59 AM, Antoine Pitrou wrote: > It would be nice if that pronouncement or decision could outline the > steps required to include an "experimental" module in the stdlib, and > the steps required to move it from "experimental" to "stable". Actually, that's a good idea - Eli, care to try your hand at writing up a counter-PEP to 408 that more explicitly documents Guido's preferred approach? It should document a standard note to be placed in the module documentation and in What's New for experimental/provisional/whatever modules. For example: "The module has been included in the standard library on a provisional basis. While major changes are not anticipated, as long as this notice remains in place, backwards incompatible changes are permitted if deemed necessary by the standard library developers. Such changes will not be made gratuitously - they will occur only if serious API flaws are uncovered that were missed prior to inclusion of the module. If the small chance of such changes is not acceptable for your use, the module is also available from PyPI with full backwards compatibility guarantees." (include direct link to module on PyPI) As far as the provisional->stable transition goes, I'd say there are a couple of options: 1. Just make it part of the normal release process to ask for each provisional module "This hasn't been causing any dramas, shall we drop the provisional warning?" 2. Explicitly create 'release blocker' tracker issues for the *next* release whenever a provisional module is added. These will basically say "either drop the provisional warning for module or bump this issue along to the next release" Former is obviously easier, latter means we're less likely to forget to do it. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 ipaddr module (was Re: PEP 408 -- Standard library __preview__ package)
On Tue, Jan 31, 2012 at 7:52 AM, Peter Moody wrote: > Note, http://pypi.python.org/pypi/ipaddr is actually the same module, > but down a few versions. I'm not sure if your concern is about the > same library having such a different api or if you had thought they > were completely different libraries. No, I knew that - my point was that the changes in the PEP 3144 branch are backwards incompatible with the existing ipaddr API (mainly due to the always-strict Network objects, with the permissive behaviour moved out to the separate Interface objects, but also due to the renamed factory functions), so it may be easier to just give the 3144 version of the module a different name. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
On 2012-01-30 20:52, Guido van Rossum wrote: TB, what's your use case for passing a float to a semaphore? Semaphores are conceptually tied to integers. You've kept arguing a few times now that the workaround you need are clumsy, but you've not explained why you're passing floats in the first place. A "fractional resource" just doesn't sound like a real use case to me. Not an example from real life and certainly not one that can't be worked around; rather a thing that caught my eyes while looking at Lib/threading.py: Say you have a "known" constant guaranteed bandwidth and you need to split it among several connections which each of them take a known fixed amount of bandwidth (no more, no less). How many connections can I reliably serve? TOTAL_BANDWIDTH/BANDWIDTH_PER_CONNECTION. Well, actually int(_)... Side note: If someone really want a discrete math implementation of a semaphore, you can replace _value with a list of resources. Then you check in acquire() "while not self._resources:" and pop a resource. In that case when a semaphore is used as a context manager it can have a useful 'as' clause. To me it seems too complicated for something that should be simple like a semaphore. Regards, TB ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
It's also potentially lossy if you incremented and decremented until integer precision is lost. My vote is for an int type check. No casting. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
On Tue, Jan 31, 2012 at 8:11 AM, Matt Joiner wrote: > It's also potentially lossy if you incremented and decremented until integer > precision is lost. My vote is for an int type check. No casting. operator.index() is built for that purpose (it's what we use these days to restrict slicing to integers). +1 for the type restriction from me. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
On Mon, Jan 30, 2012 at 2:11 PM, Matt Joiner wrote: > It's also potentially lossy if you incremented and decremented until integer > precision is lost. My vote is for an int type check. No casting. +1. Anything else is insane scope creep for something called "Semaphore". -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
2012/1/30 Nick Coghlan : > On Tue, Jan 31, 2012 at 8:11 AM, Matt Joiner wrote: >> It's also potentially lossy if you incremented and decremented until integer >> precision is lost. My vote is for an int type check. No casting. > > operator.index() is built for that purpose (it's what we use these > days to restrict slicing to integers). > > +1 for the type restriction from me. We don't need a type check. Just pass integers (obviously the only right type) to it. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Store timestamps as decimal.Decimal objects
Hi,
In issues #13882 and #11457, I propose to add an argument to functions
returning timestamps to choose the timestamp format. Python uses float
in most cases whereas float is not enough to store a timestamp with a
resolution of 1 nanosecond. I added recently time.clock_gettime() to
Python 3.3 which has a resolution of a nanosecond. The (first?) new
timestamp format will be decimal.Decimal because it is able to store
any timestamp in any resolution without loosing bits. Instead of
adding a boolean argument, I would prefer to support more formats. My
last patch provides the following formats:
- "float": float (used by default)
- "decimal": decimal.Decimal
- "datetime": datetime.datetime
- "timespec": (sec, nsec) tuple # I don't think that we need it, it
is just another example
The proposed API is:
time.time(format="datetime")
time.clock_gettime(time.CLOCK_REALTIME, format="decimal")
os.stat(path, timestamp="datetime)
etc.
This API has an issue: importing the datetime or decimal object is
implicit, I don't know if it is really an issue. (In my last patch,
the import is done too late, but it can be fixed, it is not really a
matter.)
Alexander Belopolsky proposed to use
time.time(format=datetime.datetime) instead.
--
The first step would be to add an argument to functions returning
timestamps. The second step is to accept these new formats (Decimal?)
as input, for datetime.datetime.fromtimestamp() and os.utime() for
example.
(Using decimal.Decimal, we may remove os.utimens() and use the right
function depending on the timestamp resolution.)
--
I prefer Decimal over a dummy tuple like (sec, nsec) because you can
do arithmetic on it: t2-t1, a+b, t/k, etc. It stores also the
resolution of the clock: time.time() and time.clock_gettime() have for
example different resolution (sec, ms, us for time.time() and ns for
clock_gettime()).
The decimal module is still implemented in Python, but there is
working implementation in C which is much faster. Store timestamps as
Decimal can be a motivation to integrate the C implementation :-)
--
Examples with the time module:
$ ./python
Python 3.3.0a0 (default:52f68c95e025+, Jan 26 2012, 21:54:31)
>>> import time
>>> time.time()
1327611705.948446
>>> time.time('decimal')
Decimal('1327611708.988419')
>>> t1=time.time('decimal'); t2=time.time('decimal'); t2-t1
Decimal('0.000550')
>>> t1=time.time('float'); t2=time.time('float'); t2-t1
5.9604644775390625e-06
>>> time.clock_gettime(time.CLOCK_MONOTONIC, 'decimal')
Decimal('1211833.389740312')
>>> time.clock_getres(time.CLOCK_MONOTONIC, 'decimal')
Decimal('1E-9')
>>> time.clock()
0.12
>>> time.clock('decimal')
Decimal('0.12')
Examples with os.stat:
$ ./python
Python 3.3.0a0 (default:2914ce82bf89+, Jan 30 2012, 23:07:24)
>>> import os
>>> s=os.stat("setup.py", timestamp="datetime")
>>> s.st_mtime - s.st_ctime
datetime.timedelta(0)
>>> print(s.st_atime - s.st_ctime)
52 days, 1:44:06.191293
>>> os.stat("setup.py", timestamp="timespec").st_ctime
(1323458640, 702327236)
>>> os.stat("setup.py", timestamp="decimal").st_ctime
Decimal('1323458640.702327236')
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Store timestamps as decimal.Decimal objects
Sounds good, but I also prefer Alexander's method. The type information is
already encoded in the class object. This way you don't need to maintain a
mapping of strings to classes, and other functions/third party can join in
the fun without needing access to the latest canonical mapping. Lastly
there will be no confusion or contention for duplicate keys.
On Jan 31, 2012 10:32 AM, "Victor Stinner"
wrote:
> Hi,
>
> In issues #13882 and #11457, I propose to add an argument to functions
> returning timestamps to choose the timestamp format. Python uses float
> in most cases whereas float is not enough to store a timestamp with a
> resolution of 1 nanosecond. I added recently time.clock_gettime() to
> Python 3.3 which has a resolution of a nanosecond. The (first?) new
> timestamp format will be decimal.Decimal because it is able to store
> any timestamp in any resolution without loosing bits. Instead of
> adding a boolean argument, I would prefer to support more formats. My
> last patch provides the following formats:
>
> - "float": float (used by default)
> - "decimal": decimal.Decimal
> - "datetime": datetime.datetime
> - "timespec": (sec, nsec) tuple # I don't think that we need it, it
> is just another example
>
> The proposed API is:
>
> time.time(format="datetime")
> time.clock_gettime(time.CLOCK_REALTIME, format="decimal")
> os.stat(path, timestamp="datetime)
> etc.
>
> This API has an issue: importing the datetime or decimal object is
> implicit, I don't know if it is really an issue. (In my last patch,
> the import is done too late, but it can be fixed, it is not really a
> matter.)
>
> Alexander Belopolsky proposed to use
> time.time(format=datetime.datetime) instead.
>
> --
>
> The first step would be to add an argument to functions returning
> timestamps. The second step is to accept these new formats (Decimal?)
> as input, for datetime.datetime.fromtimestamp() and os.utime() for
> example.
>
> (Using decimal.Decimal, we may remove os.utimens() and use the right
> function depending on the timestamp resolution.)
>
> --
>
> I prefer Decimal over a dummy tuple like (sec, nsec) because you can
> do arithmetic on it: t2-t1, a+b, t/k, etc. It stores also the
> resolution of the clock: time.time() and time.clock_gettime() have for
> example different resolution (sec, ms, us for time.time() and ns for
> clock_gettime()).
>
> The decimal module is still implemented in Python, but there is
> working implementation in C which is much faster. Store timestamps as
> Decimal can be a motivation to integrate the C implementation :-)
>
> --
>
> Examples with the time module:
>
> $ ./python
> Python 3.3.0a0 (default:52f68c95e025+, Jan 26 2012, 21:54:31)
> >>> import time
> >>> time.time()
> 1327611705.948446
> >>> time.time('decimal')
> Decimal('1327611708.988419')
> >>> t1=time.time('decimal'); t2=time.time('decimal'); t2-t1
> Decimal('0.000550')
> >>> t1=time.time('float'); t2=time.time('float'); t2-t1
> 5.9604644775390625e-06
> >>> time.clock_gettime(time.CLOCK_MONOTONIC, 'decimal')
> Decimal('1211833.389740312')
> >>> time.clock_getres(time.CLOCK_MONOTONIC, 'decimal')
> Decimal('1E-9')
> >>> time.clock()
> 0.12
> >>> time.clock('decimal')
> Decimal('0.12')
>
> Examples with os.stat:
>
> $ ./python
> Python 3.3.0a0 (default:2914ce82bf89+, Jan 30 2012, 23:07:24)
> >>> import os
> >>> s=os.stat("setup.py", timestamp="datetime")
> >>> s.st_mtime - s.st_ctime
> datetime.timedelta(0)
> >>> print(s.st_atime - s.st_ctime)
> 52 days, 1:44:06.191293
> >>> os.stat("setup.py", timestamp="timespec").st_ctime
> (1323458640, 702327236)
> >>> os.stat("setup.py", timestamp="decimal").st_ctime
> Decimal('1323458640.702327236')
>
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/anacrolix%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 3144 ipaddr module (was Re: PEP 408 -- Standard library __preview__ package)
Nick Coghlan writes: > 1. Draft ReStructuredText documentation for inclusion in the stdlib docs > > (still needed) No wonder people (not directly involved in development of the module) think that the proponents don't care! What good is a battery if the odds are even that you will hook it up with wrong polarity and fry your expensive components? I don't mean to criticize the proponents and mentors of *this* PEP; I recall the ipaddr vs. netaddr discussions, and clearly the API needed and got a lot of changes. That's definitely a chilling factor for writing a second document that largely covers the same material as the PEP. On the other hand, people who are not battery manufacturers have every right to use stdlib-ready documentation as a litmus test for readiness (and even if you think otherwise, you can't stop them). While you probably won't get a lot of comments from those people if you publish such docs, if you don't publish docs, you will get none. I suggest emphasizing (in the 408bis PEP that Nick suggested) the importance of documentation in convincing the "just users" audience (which is the one that stdlib is really aimed at) of the value and readiness of a module proposed for stdlib integration. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 ipaddr module (was Re: PEP 408 -- Standard library __preview__ package)
On Tue, Jan 31, 2012 at 10:51 AM, Stephen J. Turnbull wrote: > Nick Coghlan writes: > > > 1. Draft ReStructuredText documentation for inclusion in the stdlib > docs > > > > (still needed) > > No wonder people (not directly involved in development of the module) > think that the proponents don't care! What good is a battery if the > odds are even that you will hook it up with wrong polarity and fry > your expensive components? Thinking about how to document the library from a network engineer's perspective was actually the driving force behind my asking for the Address/Interface/Network split in the PEP 3144 branch. Without that, Network tries to fill both the Interface and Network role and it becomes a bit of a nightmare to write coherent prose documentation. Sure, merging them can *work* from a programming point of view, but you can't document it that way and have the API seems sensible to anyone familiar with the underlying networking concepts. Now that ReadTheDocs exists, it is of course *much* easier to draft and publish such documentation than it once was (*not-so-subtle-hint*). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Store timestamps as decimal.Decimal objects
Am 31.01.2012 00:50, schrieb Matt Joiner: > Sounds good, but I also prefer Alexander's method. The type information is > already encoded in the class object. This way you don't need to maintain a > mapping of strings to classes, and other functions/third party can join in the > fun without needing access to the latest canonical mapping. Lastly there will > be > no confusion or contention for duplicate keys. Sorry, I don't think it makes any sense to pass around classes as flags. Sure, if you do something directly with the class, it's fine, but in this case that's impossible. So you will be testing if format is datetime.datetime: ... elif format is decimal.Decimal: ... else: ... which has no advantage at all over if format == "datetime": ... elif format == "decimal": ... else: Not to speak of formats like "timespec" that don't have a respective class. And how do you propose to handle the extensibility you speak of to work? Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
stefan brunthaler, 30.01.2012 20:18: >> Well, you're aware that Python already uses threaded code where >> available? Or are you testing against Python 2? >> > Yes, and I am building on that. I assume "yes" here means "yes, I'm aware" and not "yes, I'm using Python 2", right? And you're building on top of the existing support for threaded code in order to improve it? Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
Am 30.01.2012 20:06, schrieb stefan brunthaler: >> Do you have a public repository for the code, so we can take a look? >> > I have created a patch (as Benjamin wanted) and put all of the > resources (i.e., benchmark results and the patch itself) on my home > page: > http://www.ics.uci.edu/~sbruntha/pydev.html If I read the patch correctly, most of it is auto-generated (and there is probably a few spurious changes that blow it up, such as the python-gdb.py file). But the tool that actually generates the code doesn't seem to be included? (Which means that in this form, the patch couldn't possibly be accepted.) Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Store timestamps as decimal.Decimal objects
On Tue, Jan 31, 2012 at 9:31 AM, Victor Stinner wrote: > Hi, > > In issues #13882 and #11457, I propose to add an argument to functions > returning timestamps to choose the timestamp format. Python uses float > in most cases whereas float is not enough to store a timestamp with a > resolution of 1 nanosecond. I added recently time.clock_gettime() to > Python 3.3 which has a resolution of a nanosecond. The (first?) new > timestamp format will be decimal.Decimal because it is able to store > any timestamp in any resolution without loosing bits. Instead of > adding a boolean argument, I would prefer to support more formats. I think this is definitely worth elaborating in a PEP (to recap the long discussion in #11457 if nothing else). In particular, I'd want to see a very strong case being made for supporting multiple formats over standardising on a *single* new higher precision format (for example, using decimal.Decimal in conjunction with integration of Stefan's cdecimal work) that can then be converted to other formats (like datetime) via the appropriate APIs. "There are lots of alternatives, so let's choose not to choose!" is a bad way to design an API. Helping to make decisions like this by laying out the alternatives and weighing up their costs and benefits is one of the major reasons the PEP process exists. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
