[issue1035] bytes buffer API needs to support PyBUF_LOCKDATA

[Gregory P. Smith]

New submission from Gregory P. Smith:

I've converted _bsddb.c to use the py3k buffer API for all data and keys
it takes as input.  All tests now fail with this error:

BufferError: Cannot make this object read-only.

This presumably results from this call:

  PyObject_GetBuffer(obj, view, PyBUF_LOCKDATA)

I need to lock the data so that the GIL can be released during database
operations (I/O).

Allowing bytes objects to have an immutability or readonly bit (internal
or otherwise) has been a recent topic on the python-3000 mailing list;
that would allow bytes' buffer API to satisfy this GetBuffer LOCKDATA
request...

--
components: Interpreter Core
keywords: py3k
messages: 55333
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: bytes buffer API needs to support PyBUF_LOCKDATA
type: rfe
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1036] py3k _bsddb.c patch to use the new buffer API

[Gregory P. Smith]

New submission from Gregory P. Smith:

This is my svn diff of a py3k Modules/_bsddb.c converted to use the
buffer API.  I'm submitting it here as so it doesn't get misplaced as it
currently won't work until bytes objects support PyBUF_LOCKDATA (a
separate bug)

--
assignee: gregory.p.smith
components: Extension Modules
files: py3k-_bsddb-bufferAPI-001.patch
keywords: patch, py3k
messages: 55334
nosy: gregory.p.smith
severity: normal
status: open
title: py3k _bsddb.c patch to use the new buffer API
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__Index: Modules/_bsddb.c
===
--- Modules/_bsddb.c	(revision 57549)
+++ Modules/_bsddb.c	(working copy)
@@ -348,10 +348,7 @@
 
 #define CLEAR_DBT(dbt)  (memset(&(dbt), 0, sizeof(dbt)))
 
-#define FREE_DBT(dbt)   if ((dbt.flags & (DB_DBT_MALLOC|DB_DBT_REALLOC)) && \
- dbt.data != NULL) { free(dbt.data); dbt.data = NULL; }
 
-
 static int makeDBError(int err);
 
 
@@ -372,22 +369,78 @@
 }
 
 
+/* Handy function to free a DBT and any self-allocated data within.
+   To be used on self created DBTs.  The make_dbt and make_key_dbt
+   functions have their own free routines that do more that this. */
+static void free_dbt(DBT *dbt)
+{
+if ((dbt->flags & (DB_DBT_MALLOC|DB_DBT_REALLOC)) && dbt->data != NULL) {
+ free(dbt->data);
+ dbt->data = NULL;
+}
+}
+
+
+/* Cleanup a Python buffer API view created by make_dbt() */
+static void free_buf_view(PyObject *obj, PyBuffer *view)
+{
+if (view) {
+PyObject_ReleaseBuffer(obj, view);
+PyMem_Free(view);
+}
+}
+
+
+/* Cleanup a DBT and an associated Python buffer API view
+   created by make_key_dbt() */
+#define FREE_DBT_VIEW(dbt, obj, view)\
+do { \
+free_dbt(&(dbt)); \
+free_buf_view((obj), (view)); \
+} while(0);
+
+
 /* Create a DBT structure (containing key and data values) from Python
-   strings.  Returns 1 on success, 0 on an error. */
-static int make_dbt(PyObject* obj, DBT* dbt)
+   strings.  Returns >= 1 on success, 0 on an error.  The returned_view_p
+   may be filled with a newly allocated PyBuffer view on success.
+   The caller MUST call free_buf_view() on any returned PyBuffer. */
+static int make_dbt(PyObject* obj, DBT* dbt, PyBuffer** returned_view_p)
 {
+PyBuffer *view;
+
+/* simple way to ensure the caller can detect if we've returned a
+   new buffer view or not: require their pointer to start out NULL. */
+assert(*returned_view_p == NULL);
+
 CLEAR_DBT(*dbt);
 if (obj == Py_None) {
 /* no need to do anything, the structure has already been zeroed */
 return 1;
 }
-if (!PyBytes_Check(obj)) {
+if (!PyObject_CheckBuffer(obj)) {
 PyErr_SetString(PyExc_TypeError,
-"Data values must be of type bytes or None.");
+"Data values must support the buffer API or be None.");
 return 0;
 }
-dbt->data = PyBytes_AS_STRING(obj);
-dbt->size = PyBytes_GET_SIZE(obj);
+if (!(view = PyMem_Malloc(sizeof(PyBuffer {
+PyErr_SetString(PyExc_MemoryError,
+"PyBuffer malloc failed");
+return 0;
+}
+if (PyObject_GetBuffer(obj, view, PyBUF_LOCKDATA) == -1) {
+PyMem_Free(view);
+return 0;
+}
+if (view->ndim > 1) {
+PyErr_SetString(PyExc_BufferError,
+"buffers must be single dimension");
+PyObject_ReleaseBuffer(obj, view);
+PyMem_Free(view);
+return 0;
+}
+dbt->data = view->buf;
+dbt->size = Py_SAFE_DOWNCAST(view->len, Py_ssize_t, u_int32_t);
+*returned_view_p = view;
 return 1;
 }
 
@@ -395,13 +448,20 @@
 /* Recno and Queue DBs can have integer keys.  This function figures out
what's been given, verifies that it's allowed, and then makes the DBT.
 
-   Caller MUST call FREE_DBT(key) when done. */
+   Caller MUST call FREE_DBT_VIEW(keydbt, keyobj, key_view) with all
+   returned DBT and PyBuffer values when done. */
 static int
-make_key_dbt(DBObject* self, PyObject* keyobj, DBT* key, int* pflags)
+make_key_dbt(DBObject* self, PyObject* keyobj, DBT* key, int* pflags,
+ PyBuffer** returned_view_p)
 {
 db_recno_t recno;
 int type;
+PyBuffer *view;
 
+/* simple way to ensure the caller can detect if we've returned a
+   new buffer view or not: require their pointer to start out NULL. */
+assert(*returned_view_p == NULL);
+
 CLEAR_DBT(*key);
 if (keyobj == Py_None) {
 type = _DB_get_type(self);
@@ -416,22 +476,6 @@
 /* no need to do anything, the structure has already been zeroed */
 }
 
-else if (PyBytes_Check(keyobj)) {
-/* verify access method typ

[issue1035] bytes buffer API needs to support PyBUF_LOCKDATA

[Gregory P. Smith]

Changes by Gregory P. Smith:


--
priority: normal -> 

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1037] Ill-coded identifier crashes python when coding spec is utf-8

[Hye-Shik Chang]

New submission from Hye-Shik Chang:

Illegal identifier makes python crash on UTF-8 source codes/interpreters.

Python 3.0x (py3k:57555M, Aug 27 2007, 21:23:47) 
[GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
>>> compile(b'#coding:utf-8\n\xfc', '', 'exec')
zsh: segmentation fault (core dumped)  ./python

The problem is that tokenizer.c:verify_identifer doesn't check
return value from PyUnicode_DecodeUTF8 but some invalid utf8
sequences could be there.

--
components: Unicode
keywords: py3k
messages: 55335
nosy: hyeshik.chang
priority: high
severity: normal
status: open
title: Ill-coded identifier crashes python when coding spec is utf-8
type: crash
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1202533] a bunch of infinite C recursions

[Armin Rigo]

Armin Rigo added the comment:

Assigning to Brett instead of me (according to the tracker history).

--
assignee: arigo -> brett.cannon

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1771364] Misc improvements for the io module

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks!!

Committed revision 57564.

SocketIO was recently moved to socket.py because that's the only place
that uses it.  So I've removed it from io.__all__.

The only other change I applied was related to the isatty check -- you
accidentally changed things so that buffering would *always* be set to 1
if raw.isatty is true. The intention was for this to affect the default
only.

--
resolution:  -> accepted
status: open -> closed

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1739468] Add a -z interpreter flag to execute a zip file

[Phillip J. Eby]

Phillip J. Eby added the comment:

Patch implementing an alternate approach: support automatically
importing __main__ when sys.argv[0] is an importable path.  This allows
zip files, directories, and any future importable locations (e.g. URLs)
to be used on the command line.  Note that this also means that you
don't need an option on the #! line in a zip file, which avoids hairy #!
issues on platforms like Linux (where a #! line can have at most one
argument).  __main__ is used instead of __zipmain__, since it is not
zipfile specific.

--
nosy: +pje

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1739468] Add a -z interpreter flag to execute a zip file

[Phillip J. Eby]

Changes by Phillip J. Eby:


_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1024] documentation for new SSL module

[Guido van Rossum]

Guido van Rossum added the comment:

Committed revision 57570.

I didn't review it though.

--
nosy: +gvanrossum
resolution:  -> accepted
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1739906] Add reduce to functools in 2.6

[Guido van Rossum]

Guido van Rossum added the comment:

Committed revision 57574.

--
nosy: +gvanrossum
resolution:  -> accepted
status: open -> closed

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1029] py3k: io.StringIO.getvalue() returns \r\n

[Alexandre Vassalotti]

Alexandre Vassalotti added the comment:

> That's why the current behaviour is not correct: When I write('\n'),
> getvalue() currently returns '\r\n'.

Oh, I missed your example in your initial message. So yes, I agree that
StringIO shouldn't translate newlines like that. I attached a patch that
should fix the bug.

However, I would favor removing the "newline" keyword argument, instead.

__
Tracker <[EMAIL PROTECTED]>

__Index: Lib/io.py
===
--- Lib/io.py	(revision 57506)
+++ Lib/io.py	(working copy)
@@ -1367,6 +1367,7 @@
 initial_value = str(initial_value)
 self.write(initial_value)
 self.seek(0)
+self._writetranslate = False
 
 def getvalue(self):
 self.flush()
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1038] [py3k] pdb does not work in python 3000

[Gregory P. Smith]

New submission from Gregory P. Smith:

The Lib/pdb.py debugger fails in the py3k branch.

Traceback (most recent call last):
  File "/usr/local/gps/python/py3k/Lib/pdb.py", line 1247, in main
pdb._runscript(mainpyfile)
  File "/usr/local/gps/python/py3k/Lib/pdb.py", line 1173, in _runscript
self.run(statement)
  File "/usr/local/gps/python/py3k/Lib/bdb.py", line 365, in run
exec(cmd, globals, locals)
  File "", line 1
 exec("# bla bla bla first line of your file being debugged
   ^
 SyntaxError: EOL while scanning single-quoted string
Uncaught exception. Entering post mortem debugging

--
components: Library (Lib)
keywords: py3k
messages: 55342
nosy: gregory.p.smith
severity: normal
status: open
title: [py3k] pdb does not work in python 3000
type: behavior
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1670765] email.Generator: no header wrapping for multipart/signed

[Collin Winter]

Collin Winter added the comment:

I'd like some tests demonstrating where the current implementation fails
and your patch helps, yes.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1670765] email.Generator: no header wrapping for multipart/signed

[Martin von Gagern]

Martin von Gagern added the comment:

Looks like I missed your comments on this patch.
What kind of tests do you have in mind?
Tests demonstrating where current implementation fails and my patch will
help? Or rather tests checking that this patch won't break anything
else? The former would be easy enough, but for the latter I'm not sure
how far this would go, as people tend to make strange assumptions I
would never have thought of.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1032] Improve the hackish runtime_library_dirs support for gcc

[Seo Sanghyeon]

Seo Sanghyeon added the comment:

The patch is incorrect since find returns -1 on failure. This is also a
duplicate of #1254718.

--
nosy: +sanxiyn

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1670765] email.Generator: no header wrapping for multipart/signed

[Martin von Gagern]

Martin von Gagern added the comment:

Take the attached test5.eml. Run it through the following python script:

import email
print (email.message_from_file(open("test5.eml")).as_string(False))

The result will have both instances of the X-Long-Line header rewrapped.
As the second instance is included in the digest calculation, the
signature verification will now fail.

This is a real world signature algorithm, following RFC 3156 (if I
didn't make a mistake). If you have an OpenPGP-enabled mailreader (e.g.
enigmail for Thunderbird) and have some way of injecting a mail as is
into your mail folders (e.g. a maildir-based server), then you can use
this setup to verify that the signature was correct in the first place
and is broken after parsing and reconstruction by python.

If you don't have such a setup available, and you don't believe me that
rewrapping the header breaks the signature, then I could either devise
some unrealistic but easy-to-check signing process, or could try to get
this working with an S/MIME signature using an X.509 certificate. I
would rather avoid this, though.

_
Tracker <[EMAIL PROTECTED]>

_--- Begin Message ---
This is the signed contents.


signature.asc
Description: OpenPGP digital signature
--- End Message ---
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1528802] Turkish Character

[Ismail Donmez]

Ismail Donmez added the comment:

This works fine with python 2.4 :

>>> import locale
>>> locale.setlocale(locale.LC_ALL,"tr_TR.UTF-8")
'tr_TR.UTF-8'
>>> print u"Mayıs".upper()
MAYIS

--
nosy: +cartman

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue459007] Document sys.path on Windows

[Sean Reifschneider]

Sean Reifschneider added the comment:

This is a high priority item that hasn't been touched in half a decade.
 Mark: Consider this a reminder if this is still on your agenda to do. 
If not, shall we just close it?  Is there anyone else that could do this?

--
nosy: +jafo


Tracker <[EMAIL PROTECTED]>


___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1033] Support for newline and encoding in tempfile module

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks!

Committed revision 57594.

General note: please run Tools/scripts/reindent.py over your files after
editing so I won't have to.

--
nosy: +gvanrossum
resolution:  -> accepted
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1774736] Binding fails

[Ali Gholami Rudi]

Changes by Ali Gholami Rudi:


--
versions: +Python 3.0

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1039] Asssertion in Windows debug build

[Thomas Heller]

New submission from Thomas Heller:

In a windows debug build, an assertion is triggered when os.execvpe is
called with an empty argument list:

self.assertRaises(OSError, os.execvpe, 'no such app-', [], None)

The same problem is present in the trunk version.
Attached is a patch that fixes this, with a test.

--
components: Windows
files: os.diff
messages: 55350
nosy: theller
severity: normal
status: open
title: Asssertion in Windows debug build
type: crash
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__Index: Lib/test/test_os.py
===
--- Lib/test/test_os.py	(revision 57596)
+++ Lib/test/test_os.py	(working copy)
@@ -441,6 +441,9 @@
 def test_execvpe_with_bad_program(self):
 self.assertRaises(OSError, os.execvpe, 'no such app-', [], None)
 
+def test_execvpe_with_bad_arglist(self):
+self.assertRaises(ValueError, os.execvpe, 'notepad', [], None)
+
 class Win32ErrorTests(unittest.TestCase):
 def test_rename(self):
 self.assertRaises(WindowsError, os.rename, test_support.TESTFN, test_support.TESTFN+".bak")
Index: Modules/posixmodule.c
===
--- Modules/posixmodule.c	(revision 57596)
+++ Modules/posixmodule.c	(working copy)
@@ -2834,6 +2834,11 @@
 PyMem_Free(path);
 		return NULL;
 	}
+	if (argc < 1) {
+		PyErr_SetString(PyExc_ValueError, "execv() arg 2 must not be empty");
+PyMem_Free(path);
+		return NULL;
+	}
 
 	argvlist = PyMem_NEW(char *, argc+1);
 	if (argvlist == NULL) {
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1043] test_builtin failure on Windows

[Thomas Heller]

New submission from Thomas Heller:

test test_builtin failed -- Traceback (most recent call last):
  File "c:\svn\py3k\lib\test\test_builtin.py", line 1473, in test_round
self.assertEqual(round(1e20), 1e20)
AssertionError: 0 != 1e+020

--
components: Windows
messages: 55355
nosy: theller
severity: normal
status: open
title: test_builtin failure on Windows
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1042] test_glob fails with UnicodeDecodeError

[Thomas Heller]

New submission from Thomas Heller:

Unicode errors in various tests - not only in test_glob:

  test_glob
  test test_glob failed -- Traceback (most recent call last):
File "c:\svn\py3k\lib\test\test_glob.py", line 87, in
test_glob_directory_names
  eq(self.glob('*', '*a'), [])
File "c:\svn\py3k\lib\test\test_glob.py", line 41, in glob
  res = glob.glob(p)
File "c:\svn\py3k\lib\glob.py", line 16, in glob
  return list(iglob(pathname))
File "c:\svn\py3k\lib\glob.py", line 42, in iglob
  for name in glob_in_dir(dirname, basename):
File "c:\svn\py3k\lib\glob.py", line 56, in glob1
  names = os.listdir(dirname)
  UnicodeDecodeError: 'utf8' codec can't decode bytes in position 27-31:
unexpected end of data

--
components: Windows
messages: 55354
nosy: theller
severity: normal
status: open
title: test_glob fails with UnicodeDecodeError
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1040] Unicode problem with TZ

[Thomas Heller]

Thomas Heller added the comment:

BTW, setting the environment variable TZ to, say, 'GMT' makes the
problem go away.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1040] Unicode problem with TZ

[Thomas Heller]

New submission from Thomas Heller:

In my german version of winXP SP2, python3 cannot import the time module:

c:\svn\py3k\PCbuild>python_d
Python 3.0x (py3k:57600M, Aug 28 2007, 07:58:23) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 9-11:
invalid data
[36719 refs]
>>> ^Z

The problem is that the libc '_tzname' variable contains umlauts.  For
comparison, here is what Python2.5 does:

c:\svn\py3k\PCbuild>\python25\python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.tzname
('Westeurop\xe4ische Normalzeit', 'Westeurop\xe4ische Normalzeit')
>>>

--
components: Windows
messages: 55351
nosy: theller
severity: normal
status: open
title: Unicode problem with TZ
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1041] io.py problems on Windows

[Thomas Heller]

New submission from Thomas Heller:

Running the PCBuild\rt.bat script fails when it compares the expected output
with the actual output.  Some inspection shows that the comparison fails
because
there are '\n' linefeeds in the expected and '\n\r' linefeeds in the
actual output:

  c:\svn\py3k\PCbuild>python_d  -E -tt ../lib/test/regrtest.py
  test_grammar
  test test_grammar produced unexpected output:
  **
  *** mismatch between line 1 of expected output and line 1 of actual
output:
  - test_grammar
  + test_grammar
  ? +
  (['test_grammar\n'], ['test_grammar\r\n'])
  ... and so on ...

(The last line is printed by some code I added to Lib\regrtest.py.)

It seems that this behaviour was introduced by r57186:

  New I/O code from Tony Lownds implement newline feature correctly,
  and implements .newlines attribute in a 2.x-compatible fashion.


The patch at http://bugs.python.org/issue1029 apparently fixes this problem.

--
components: Windows
messages: 55353
nosy: theller
severity: normal
status: open
title: io.py problems on Windows
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com