Re: [Python-Dev] [Python-checkins] cpython: Issue #15031: Refactor some code in importlib pertaining to validating
Nice improvement. Just a couple of minor cleanup suggestions.
On Sat, Jan 12, 2013 at 9:09 AM, brett.cannon
wrote:
> +else:
> +# To prevent having to make all messages have a conditional name.
> +name = 'bytecode'
For consistency with other default/implied names, I suggest wrapping
this in angle brackets: "")
> +if path is not None:
> +exc_details['path'] = path
> +magic = data[:4]
> +raw_timestamp = data[4:8]
> +raw_size = data[8:12]
> +if magic != _MAGIC_BYTES:
> +msg = 'bad magic number in {!r}: {!r}'.format(name, magic)
> +raise ImportError(msg, **exc_details)
> +elif len(raw_timestamp) != 4:
> +message = 'bad timestamp in {!r}'.format(name)
> +_verbose_message(message)
> +raise EOFError(message)
> +elif len(raw_size) != 4:
> +message = 'bad size in {!r}'.format(name)
> +_verbose_message(message)
> +raise EOFError(message)
For timestamp and size "incomplete" would probably be a better word
than "bad" in the error messages (since we're only checking the length
rather than the value).
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (aef7db0d3893): sum=287
Am 11.01.2013 18:19, schrieb Andrea Griffini: > On Fri, Jan 11, 2013 at 5:08 PM, Christian Heimes > wrote: >> It has more issues. Coverity has sent me some complains, see attachment. > > The second complaint seems a false positive; if self->extra is null > then children is set to 0 and following code is not executed. Yes, Coverity's results have lots of false positives. On the other hand it's able to detect hard to find resource leaks. You get used to the false positives and learn to appreciate the good stuff. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Daily reference leaks (aef7db0d3893): sum=287
On Thu, Jan 10, 2013 at 10:09 PM, Nick Coghlan wrote: > On Fri, Jan 11, 2013 at 2:57 PM, wrote: > > results for aef7db0d3893 on branch "default" > > > > > > test_dbm leaked [2, 0, 0] references, sum=2 > > test_dbm leaked [2, 2, 1] memory blocks, sum=5 > > Hmm, I'm starting to wonder if there's something to this one - it > seems to be popping up a bit lately. > > > test_xml_etree_c leaked [56, 56, 56] references, sum=168 > > test_xml_etree_c leaked [36, 38, 38] memory blocks, sum=112 > > I'm gonna take a wild guess and suggest there may be a problem with > the recent pickling fix in the C extension :) > Yep. We're on it :-) Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (aef7db0d3893): sum=287
On Fri, Jan 11, 2013 at 5:08 PM, Christian Heimes wrote: > It has more issues. Coverity has sent me some complains, see attachment. The second complaint seems a false positive; if self->extra is null then children is set to 0 and following code is not executed. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Issue #15539: Fix a number of bugs in Tools/scripts/pindent.py.
This seems to have caused the Windows buildbots to fail.
On Fri, Jan 11, 2013 at 5:40 AM, serhiy.storchaka
wrote:
> http://hg.python.org/cpython/rev/8452c23139c6
> changeset: 81407:8452c23139c6
> parent: 81399:5ec8daab477a
> parent: 81406:01df1f7841b2
> user:Serhiy Storchaka
> date:Fri Jan 11 12:12:32 2013 +0200
> summary:
> Issue #15539: Fix a number of bugs in Tools/scripts/pindent.py.
> Now pindent.py works with a "with" statement. pindent.py no longer produces
> improper indentation. pindent.py now works with continued lines broken after
> "class" or "def" keywords and with continuations at the start of line. Added
> regression tests for pindent.py. Modernized pindent.py.
>
> files:
> Lib/test/test_tools.py | 326 ++-
> Misc/NEWS|7 +
> Tools/scripts/pindent.py | 166 +
> 3 files changed, 393 insertions(+), 106 deletions(-)
>
>
> diff --git a/Lib/test/test_tools.py b/Lib/test/test_tools.py
> --- a/Lib/test/test_tools.py
> +++ b/Lib/test/test_tools.py
> @@ -9,10 +9,13 @@
> import importlib.machinery
> import unittest
> from unittest import mock
> +import shutil
> +import subprocess
> import sysconfig
> import tempfile
> +import textwrap
> from test import support
> -from test.script_helper import assert_python_ok
> +from test.script_helper import assert_python_ok, temp_dir
>
> if not sysconfig.is_python_build():
> # XXX some installers do contain the tools, should we detect that
> @@ -36,6 +39,327 @@
> self.assertGreater(err, b'')
>
>
> +class PindentTests(unittest.TestCase):
> +script = os.path.join(scriptsdir, 'pindent.py')
> +
> +def assertFileEqual(self, fn1, fn2):
> +with open(fn1) as f1, open(fn2) as f2:
> +self.assertEqual(f1.readlines(), f2.readlines())
> +
> +def pindent(self, source, *args):
> +with subprocess.Popen(
> +(sys.executable, self.script) + args,
> +stdin=subprocess.PIPE, stdout=subprocess.PIPE,
> +universal_newlines=True) as proc:
> +out, err = proc.communicate(source)
> +self.assertIsNone(err)
> +return out
> +
> +def lstriplines(self, data):
> +return '\n'.join(line.lstrip() for line in data.splitlines()) + '\n'
> +
> +def test_selftest(self):
> +with temp_dir() as directory:
> +data_path = os.path.join(directory, '_test.py')
> +with open(self.script) as f:
> +closed = f.read()
> +with open(data_path, 'w') as f:
> +f.write(closed)
> +
> +rc, out, err = assert_python_ok(self.script, '-d', data_path)
> +self.assertEqual(out, b'')
> +self.assertEqual(err, b'')
> +backup = data_path + '~'
> +self.assertTrue(os.path.exists(backup))
> +with open(backup) as f:
> +self.assertEqual(f.read(), closed)
> +with open(data_path) as f:
> +clean = f.read()
> +compile(clean, '_test.py', 'exec')
> +self.assertEqual(self.pindent(clean, '-c'), closed)
> +self.assertEqual(self.pindent(closed, '-d'), clean)
> +
> +rc, out, err = assert_python_ok(self.script, '-c', data_path)
> +self.assertEqual(out, b'')
> +self.assertEqual(err, b'')
> +with open(backup) as f:
> +self.assertEqual(f.read(), clean)
> +with open(data_path) as f:
> +self.assertEqual(f.read(), closed)
> +
> +broken = self.lstriplines(closed)
> +with open(data_path, 'w') as f:
> +f.write(broken)
> +rc, out, err = assert_python_ok(self.script, '-r', data_path)
> +self.assertEqual(out, b'')
> +self.assertEqual(err, b'')
> +with open(backup) as f:
> +self.assertEqual(f.read(), broken)
> +with open(data_path) as f:
> +indented = f.read()
> +compile(indented, '_test.py', 'exec')
> +self.assertEqual(self.pindent(broken, '-r'), indented)
> +
> +def pindent_test(self, clean, closed):
> +self.assertEqual(self.pindent(clean, '-c'), closed)
> +self.assertEqual(self.pindent(closed, '-d'), clean)
> +broken = self.lstriplines(closed)
> +self.assertEqual(self.pindent(broken, '-r', '-e', '-s', '4'), closed)
> +
> +def test_statements(self):
> +clean = textwrap.dedent("""\
> +if a:
> +pass
> +
> +if a:
> +pass
> +else:
> +pass
> +
> +if a:
> +pass
> +elif:
> +pass
> +else:
> +pass
> +
> +while a:
> +break
> +
> +while a:
> +break
> +else:
> +pass
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-01-04 - 2013-01-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open3825 ( +8) closed 24888 (+65) total 28713 (+73) Open issues with patches: 1668 Issues opened (41) == #12944: Accept arbitrary files for packaging's upload command http://bugs.python.org/issue12944 reopened by eric.araujo #13198: Remove duplicate definition of write_record_file http://bugs.python.org/issue13198 reopened by eric.araujo #15539: Fixing Tools/scripts/pindent.py http://bugs.python.org/issue15539 reopened by serhiy.storchaka #16076: xml.etree.ElementTree.Element is no longer pickleable http://bugs.python.org/issue16076 reopened by ezio.melotti #16823: Python crashes on running tkinter code with threads http://bugs.python.org/issue16823 reopened by terry.reedy #16864: sqlite3.Cursor.lastrowid isn't populated when executing a SQL http://bugs.python.org/issue16864 opened by jim_minter #16865: ctypes arrays >=2GB in length causes exception http://bugs.python.org/issue16865 opened by coderforlife #16866: libainstall doesn't create $(BINDIR) directory http://bugs.python.org/issue16866 opened by bennoleslie #16869: makesetup should support .S source files http://bugs.python.org/issue16869 opened by bennoleslie #16878: argparse: positional args with nargs='*' defaults to [] http://bugs.python.org/issue16878 opened by chris.jerdonek #16879: distutils.command.config uses fragile constant temporary file http://bugs.python.org/issue16879 opened by mgorny #16880: Importing "imp" will fail if dynamic loading not supported http://bugs.python.org/issue16880 opened by Jeffrey.Armstrong #16885: SQLite3 iterdump ordering http://bugs.python.org/issue16885 opened by Jamie.Spence #16886: Doctests in test_dictcomp depend on dict order http://bugs.python.org/issue16886 opened by fwierzbicki #16887: IDLE - tabify/untabify applied when clicking Cancel http://bugs.python.org/issue16887 opened by serwy #16889: facilitate log output starting at beginning of line http://bugs.python.org/issue16889 opened by chris.jerdonek #16891: Fix docs about module search order http://bugs.python.org/issue16891 opened by dmugtasimov #16892: Windows bug picking up stdin from a pipe http://bugs.python.org/issue16892 opened by oeffner #16893: Create IDLE help.txt from Doc/library/idle.rst http://bugs.python.org/issue16893 opened by zach.ware #16894: Function attribute access doesn't invoke methods in dict subcl http://bugs.python.org/issue16894 opened by dabeaz #16895: Batch file to mimic 'make' on Windows http://bugs.python.org/issue16895 opened by zach.ware #16899: Add support for C99 complex type (_Complex) as ctypes.c_comple http://bugs.python.org/issue16899 opened by rutsky #16901: In http.cookiejar.FileCookieJar() the .load() and .revert() me http://bugs.python.org/issue16901 opened by py.user #16902: Add OSS module support for Solaris http://bugs.python.org/issue16902 opened by brian-cameron-oracle #16903: subprocess.Popen.communicate with universal_newlines=True does http://bugs.python.org/issue16903 opened by serhiy.storchaka #16904: Avoid unnecessary and possibly unsafe code from http.client.HT http://bugs.python.org/issue16904 opened by sanyi #16907: Distutils fails to build extension in path with spaces http://bugs.python.org/issue16907 opened by klo.uo #16909: urlparse: add userinfo attribute http://bugs.python.org/issue16909 opened by olof #16914: timestamping in smtplib.py to help troubleshoot server timeout http://bugs.python.org/issue16914 opened by gac #16915: mode of socket.makefile is more limited than documentation sug http://bugs.python.org/issue16915 opened by Antoon.Pardon #16916: The Extended Iterable Unpacking (PEP-3132) for byte strings do http://bugs.python.org/issue16916 opened by marco.buttu #16921: Docs of subprocess.CREATE_NEW_CONSOLE are wrong http://bugs.python.org/issue16921 opened by torsten #16923: test_ssl kicks up a lot of ResourceWarnings http://bugs.python.org/issue16923 opened by benjamin.peterson #16926: setup.py register does not always respect --repository http://bugs.python.org/issue16926 opened by chris.jerdonek #16927: Separate built-in types from functions and group similar funct http://bugs.python.org/issue16927 opened by ezio.melotti #16930: mention limitations and/or alternatives to hg graft http://bugs.python.org/issue16930 opened by chris.jerdonek #16931: mention work-around to create diffs in default/non-git mode http://bugs.python.org/issue16931 opened by chris.jerdonek #16933: argparse: remove magic from examples http://bugs.python.org/issue16933 opened by guettli #16935: unittest should understand SkipTest at import time during test http://bugs.python.org/issue16935 opened by brett.cannon #16936: Documentation for stat.S_IFMT inconsistent http://bugs.python
Re: [Python-Dev] fork or exec?
> *Lots* of applications make use of POSIX semantics for fork() / exec(). This doesn't mean much. We're talking about inheritance of FDs > 2 upon exec, which is a very limited subset of "POSIX semantics for fork() / exec()". I personally think that there's been enough feedback to show that we should stick with the default POSIX behavior, however broken it is... > Can someone please point to a writeop of the security issues involved? I've posted sample codes earlier in this thread, but here's a writeup by Ulrich Drepper: http://udrepper.livejournal.com/20407.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (aef7db0d3893): sum=287
Am 11.01.2013 07:09, schrieb Nick Coghlan:
> On Fri, Jan 11, 2013 at 2:57 PM, wrote:
>> results for aef7db0d3893 on branch "default"
>>
>>
>> test_dbm leaked [2, 0, 0] references, sum=2
>> test_dbm leaked [2, 2, 1] memory blocks, sum=5
>
> Hmm, I'm starting to wonder if there's something to this one - it
> seems to be popping up a bit lately.
>
>> test_xml_etree_c leaked [56, 56, 56] references, sum=168
>> test_xml_etree_c leaked [36, 38, 38] memory blocks, sum=112
>
> I'm gonna take a wild guess and suggest there may be a problem with
> the recent pickling fix in the C extension :)
It has more issues. Coverity has sent me some complains, see attachment.
--- Begin Message ---
Hi,
Please find the latest report on new defect(s) introduced to Python found with
Coverity SCAN
Defect(s) Reported-by: Coverity Scan
** CID 967027: Dereference null return value (NULL_RETURNS)
/Modules/_elementtree.c: 966
http://scan5.coverity.com:8080//sourcebrowser.htm?projectId=10226#mergedDefectId=967027
** CID 967026: Dereference after null check (FORWARD_NULL)
/Modules/_elementtree.c: 841
http://scan5.coverity.com:8080//sourcebrowser.htm?projectId=10226#mergedDefectId=967026
CID 967027: Dereference null return value (NULL_RETURNS)
/Modules/_elementtree.c: 963 ( returned_null)
960
961/* More instance dict members than we know to handle? */
962tag = attrib = text = tail = children = NULL;
>>> Function "PyTuple_New(Py_ssize_t)" returns null (checked 169 out of 190
>>> times).
963args = PyTuple_New(0);
964error = ! PyArg_ParseTupleAndKeywords(args, state, "|$O",
kwlist, &tag,
965 &attrib, &text, &tail,
&children);
966Py_DECREF(args);
967if (error)
/Modules/_elementtree.c: 963 ( var_assigned)
960
961/* More instance dict members than we know to handle? */
962tag = attrib = text = tail = children = NULL;
>>> Assigning: "args" = null return value from "PyTuple_New(Py_ssize_t)".
963args = PyTuple_New(0);
964error = ! PyArg_ParseTupleAndKeywords(args, state, "|$O",
kwlist, &tag,
965 &attrib, &text, &tail,
&children);
966Py_DECREF(args);
967if (error)
/Modules/_elementtree.c: 966 ( dereference)
963args = PyTuple_New(0);
964error = ! PyArg_ParseTupleAndKeywords(args, state, "|$O",
kwlist, &tag,
965 &attrib, &text, &tail,
&children);
>>> CID 967027: Dereference null return value (NULL_RETURNS)
>>> Dereferencing a null pointer "args".
966Py_DECREF(args);
967if (error)
968return NULL;
969else
970return element_setstate_from_attributes(self, tag, attrib,
text,
CID 967026: Dereference after null check (FORWARD_NULL)
/Modules/_elementtree.c: 837 ( var_compare_op)
834PyObject *instancedict = NULL, *children;
835
836/* Build a list of children. */
>>> Comparing "self->extra" to null implies that "self->extra" might be null.
837children = PyList_New(self->extra ? self->extra->length : 0);
838if (!children)
839return NULL;
840for (i = 0; i < PyList_GET_SIZE(children); i++) {
841PyObject *child = self->extra->children[i];
/Modules/_elementtree.c: 841 ( var_deref_op)
838if (!children)
839return NULL;
840for (i = 0; i < PyList_GET_SIZE(children); i++) {
>>> CID 967026: Dereference after null check (FORWARD_NULL)
>>> Dereferencing null pointer "self->extra".
841PyObject *child = self->extra->children[i];
842Py_INCREF(child);
843PyList_SET_ITEM(children, i, child);
844}
845
To view the defects in Coverity Scan visit, http://scan5.coverity.com:8080
If you don't have a username, you can request one by emailing:
scan-ad...@coverity.com
To unsubscribe from the email notification for new defects,
http://scan.coverity.com/email_unsubscribe.html
If you are project owner, you can subscribe your team member by accessing
http://scan.coverity.com/email_subscription.html
--- End Message ---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] fork or exec?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/10/2013 02:55 PM, Charles-François Natali wrote: > Indeed, it should be really rare. *Lots* of applications make use of POSIX semantics for fork() / exec(). > There are far more programs that are bitten by FD inheritance upon > exec than programs relying on it, and whereas failures and security > issues in the first category are hard to debug and unpredictable > (especially in a multi-threaded program), Can someone please point to a writeop of the security issues involved? Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlDwMtsACgkQ+gerLs4ltQ7e7QCfdc1yxE+1iF6Gf/O9reOxStIG V8YAoKs79z5DHA3dyTDI+9yWkSSW0VD6 =9FVK -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] fork or exec?
On Jan 11, 2013, at 06:25 PM, Ben Leslie wrote: >Python is not UNIX, but I think if you are directly using the POSIX >interfaces they should work (more or less) the same way the would if you were >writing a C program. (Some of us still use Python to prototype things that >will later be converted to C!). +1 Or put it another way: Python isn't POSIX but it exposes POSIX APIs, they should act like POSIX APIs. It's perfectly fine to provide alternative APIs that have different behavior. I think it's exactly analogous to Python exposing Windows or OS X specific APIs. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.3): #16919: test_crypt now works with unittest test discovery. Patch by Zachary
On Fri, Jan 11, 2013 at 8:37 AM, R. David Murray wrote:
> On Fri, 11 Jan 2013 08:11:00 +0100, Antoine Pitrou
> wrote:
>> On Fri, 11 Jan 2013 04:20:21 +0100 (CET)
>> ezio.melotti wrote:
>> >
>> > diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py
>> > --- a/Lib/test/test_crypt.py
>> > +++ b/Lib/test/test_crypt.py
>> > @@ -1,7 +1,11 @@
>> > from test import support
>> > import unittest
>> >
>> > -crypt = support.import_module('crypt')
>> > +def setUpModule():
>> > +# this import will raise unittest.SkipTest if _crypt doesn't exist,
>> > +# so it has to be done in setUpModule for test discovery to work
>> > +global crypt
>> > +crypt = support.import_module('crypt')
>>
>> Yikes.
>> Couldn't unittest support SkipTest being raised at import instead?
>> setUpModule is an ugly way to do this.
>
> Indeed. Almost every use of support.import_module will have this issue,
> so fixing unittest is by far the better fix.
Bug filed: http://bugs.python.org/issue16935
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.3): #16919: test_crypt now works with unittest test discovery. Patch by Zachary
On Fri, 11 Jan 2013 08:11:00 +0100, Antoine Pitrou wrote:
> On Fri, 11 Jan 2013 04:20:21 +0100 (CET)
> ezio.melotti wrote:
> >
> > diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py
> > --- a/Lib/test/test_crypt.py
> > +++ b/Lib/test/test_crypt.py
> > @@ -1,7 +1,11 @@
> > from test import support
> > import unittest
> >
> > -crypt = support.import_module('crypt')
> > +def setUpModule():
> > +# this import will raise unittest.SkipTest if _crypt doesn't exist,
> > +# so it has to be done in setUpModule for test discovery to work
> > +global crypt
> > +crypt = support.import_module('crypt')
>
> Yikes.
> Couldn't unittest support SkipTest being raised at import instead?
> setUpModule is an ugly way to do this.
Indeed. Almost every use of support.import_module will have this issue,
so fixing unittest is by far the better fix.
--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] fork or exec?
On 1/11/2013 2:25 AM, Ben Leslie wrote: Python is not UNIX, but I think if you are directly using the POSIX interfaces they should work (more or less) the same way the would if you were writing a C program. (Some of us still use Python to prototype things that will later be converted to C!). I believe it has been the intent that if xxx is s syscall, then os.xxx should be a thin wrapper with the same defaults. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
