[issue31871] Support for file descriptor params in os.path

[Mateusz Kurek]

New submission from Mateusz Kurek :

Since Python 3.3, some os module functions, like os.stat 
(https://docs.python.org/3/library/os.html#os.stat), support passing file 
descriptor instead of a path. os.path functions, on the other hand (like 
os.path.exists - https://docs.python.org/3/library/os.path.html#os.path.exists 
- or os.path.samefile - 
https://docs.python.org/3/library/os.path.html#os.path.samefile) mention using 
os.stat underneath, but according to documentation, does not support passing 
file descriptor instead of a path (at least it's not mentioned in the docs that 
this feature is supported). Is this intentional (os.path functions should not 
take file descriptor params) or this feature is officialy supported, but it's 
ommited in the docs?

--
assignee: docs@python
components: Documentation
messages: 305023
nosy: Mateusz Kurek, docs@python
priority: normal
severity: normal
status: open
title: Support for file descriptor params in os.path
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20486] msilib: can't close opened database

[Berker Peksag]

Berker Peksag  added the comment:

xoviat, would you like to send your patch as a pull request on GitHub? It would 
be nice to add a simple test that the new Close() works correctly. I can do 
that if you don't have time, thank you!

Steve, can we apply this to bugfix branches?

--
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDLE doesn't recognise installed packages

[Daniel Tangemann]

ok, I did that. I noticed that this path: 
'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' 
is missing when I run the python.exe without IDLE. how do I fix this?
also I get a syntax error when I try that:
"To make sure you are running pip with the same binary
as IDLE, enter path-to-binary -m pip  C:\Programs\Python37\python.exe -m pip list"
I need to have this explained with baby steps, I'm sure I'm just doing 
something wrong there.

thank you for the help! I really appreciate it!

> Terry Reedy  hat am 24. Oktober 2017 um 08:36 geschrieben:
>
>
> On 10/23/2017 10:23 AM, Daniel Tangemann wrote:
> > I've recently downloaded and installed python 3.6. (I had already also 2.7 
> > and 3.2 on my computer) Initially pip was looking in the wrong directory to 
> > install to, so I changed that. then it had trouble installing matplotlib, 
> > so I decided to get rid of the older versions of python, which srewed 
> > things up even more. now scrips that I had written (in 3.6), that were 
> > running without errors before, aren't working anymore. I tried reinstalling 
> > python, and I tried the repair option multiple times as well. when I look 
> > into the python folder, I can see the modules that I have installed (and 
> > that I import into those scripts), but the IDLE doesn't see them! what's 
> > even more weird, is that "pip list" doesn't bring up anything but pip 
> > itself, while typing "pip install matplotlib" returns a message that
> > it's already installed. how do I fix this?
> > cheers
>
> Recognition of installed packages is done by the python running IDLE and
> executing your import statements, by not IDLE. The only effect IDLE
> could have is any manipulation of sys.path.
>
> You can find the executable running IDLE with
>
> >>> import sys; sys.executable
> 'C:\\Programs\\Python37\\pythonw.exe'
>
> Find the sys.path being used with
> >>> sys.path
>
> If you run the same binary (minus the 'w' if present), you can find the
> sys.path used without IDLE. You can also test imports without IDLE in use.
>
> It is possible that you have more than one binary around, but I cannot
> tell from here. To make sure you are running pip with the same binary
> as IDLE, enter path-to-binary -m pip  instance, on windows, given the above
>
> path> C:\Programs\Python37\python.exe -m pip list
>
> --
> Terry Jan Reedy
>
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31870] add timeout parameter for get_server_certificate in ssl.py

[Vex Woo]

New submission from Vex Woo :

The original get_server_certificate in ssl.py does not support socket timeout,

def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None):
"""Retrieve the certificate from the server at the specified address,
and return it as a PEM-encoded string.
If 'ca_certs' is specified, validate the server cert against it.
If 'ssl_version' is specified, use it in the connection attempt."""

host, port = addr
if ca_certs is not None:
cert_reqs = CERT_REQUIRED
else:
cert_reqs = CERT_NONE
context = _create_stdlib_context(ssl_version,
 cert_reqs=cert_reqs,
 cafile=ca_certs)
with  create_connection(addr) as sock:
with context.wrap_socket(sock) as sslsock:
dercert = sslsock.getpeercert(True)
return DER_cert_to_PEM_cert(dercert)

If a timeout parameter, a sample demo can be here:

>>> import ssl
>>> ssl.get_server_certificate(("www.qq.com", 443), timeout=6)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.7/ssl.py", line 1017, in get_server_certificate
with closing(create_connection(addr, timeout)) as sock:
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
raise err
socket.error: [Errno 101] Network is unreachable

--
components: Library (Lib)
files: ssl.py
messages: 305021
nosy: Nixawk
priority: normal
pull_requests: 4092
severity: normal
status: open
title: add timeout parameter for get_server_certificate in ssl.py
type: enhancement
Added file: https://bugs.python.org/file47238/ssl.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31863] Inconsistent returncode/exitcode for terminated child processes on Windows

[Eryk Sun]

Change by Eryk Sun :


--
stage:  -> needs patch
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31863] Inconsistent returncode/exitcode for terminated child processes on Windows

[Eryk Sun]

Eryk Sun  added the comment:

Setting the exit code to the negative of a C signal value isn't generally 
meaningful in Windows. It seems multiprocessing doesn't have a significant use 
for this, other than getting a formatted exit code in the repr via its 
_exitcode_to_name dict. For example:

p = multiprocessing.Process(target=time.sleep, args=(30,))
p.start()
p.terminate()

>>> p


This may mislead people into thinking incorrectly that Windows implements POSIX 
signals. Python uses the C runtime's emulation of the basic set of required 
signals. SIGSEGV, SIGFPE, and SIGILL are based on exceptions. SIGINT and 
SIGBREAK are based on console control events. SIGABRT and SIGTERM are for use 
with C `raise`. Additionally it implements os.kill via TerminateProcess and 
GenerateConsoleCntrlEvent. (The latter takes process group IDs, so it should 
have been used to implement os.killpg instead. Its use in os.kill is wrong and 
confusing.)

The normal exit code for a forced shutdown is 1, which you can confirm via Task 
Manager or `taskkill /F`. subprocess is correct here. I think multiprocessing 
should follow suit.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Nick Coghlan]

Nick Coghlan  added the comment:

Serhiy's PR avoids the cryptic BytesWarning on Py3 while minimising the 
overhead of the new typecheck, so I've closed Berker's PR in favour of that one 
(which now has approved reviews from both Brett and I, so Serhiy will merge it 
when he's ready to do so).

Thanks for both patches!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Determine the container class of an object in Python 3

[Steve D'Aprano]

On Thu, 26 Oct 2017 12:25 pm, qrious wrote:

> Class1 is instantiated in Class2 as follows. Class2 also contains another
> variable, say:
> 
> class Class2:
> class1 = Class1()
> a = 0
> 
> I want to create a method myDef() in Class1 that can read or write to a. How
> do I access a from within myDef() to access a?

The only way is to hard-code the name "Class2" in myDef.

But the usual way to fix this problem is:


class Class1:
def __init__(self, owner):
self.owner = owner
def myDef(self):
return self.owner.a


def Class2:
def __init__(self):
self.class1 = Class1(self)
self.a = 0

Technically, it is a circular reference, but don't worry about it, Python's
garbage collector can resolve such circular references. If it becomes a
problem -- and it shouldn't, but if it does -- you can use a weakref. But
seriously, don't worry about it unless you need to.

Another solution is to make Class1 a mixin, and then rely on inheritence:

class MyMixin:
def myDef(self):
return self.a

class Class2(MyMixin):
def __init__(self):
self.a = 0



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

[Steve D'Aprano]

On Thu, 26 Oct 2017 08:22 am, danceswithnumb...@gmail.com wrote:

> with each pass you can compress untill the entropy is so random it can no
> longer be comressed.


Which is another way of saying that you cannot compress random binary data.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Determine the container class of an object in Python 3

[Chris Angelico]

On Thu, Oct 26, 2017 at 12:25 PM, qrious  wrote:
>
> Class1 is instantiated in Class2 as follows. Class2 also contains another 
> variable, say:
>
> class Class2:
> class1 = Class1()
> a = 0
>
> I want to create a method myDef() in Class1 that can read or write to a. How 
> do I access a from within myDef() to access a?
>
> Calling Class2.a is not an option as Class1 does not have any knowledge about 
> its container class a priori. Also, this will hardcode the path. I want to 
> know of a built-in method (kind of equivalent of super() for inheritance) 
> that will give me the container class reference, Class2 in this case.
>
> Thanks.

There isn't any.

Relationships like this are one-way; you can go from Class2 to its
attribute named "class1", which is an instance of Class1, but you
can't go back the other way. There could be multiple ways to get to
that same object, so there's no way for it to know which one you want.

Now, there MAY be some strange and arcane magic that you can do as you
construct that object to figure out what class it's in. But generally,
you should avoid that kind of thing. There is (almost certainly) a
better way to accomplish whatever it is you're aiming for.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue24920] shutil.get_terminal_size throws AttributeError

[Terry J. Reedy]

Terry J. Reedy  added the comment:

If one starts IDLE from a command-line console (python -m idlelib) or Python 
console (import idlelib.idle), sys.__stdout__ is the TextIOWraper for that 
console and .fileno() returns 1.  .get_terminal_size() will then return the 
console size.  The exception occurs when IDLE is  started from an icon.

Implementing David's suggestion for shutil will be easy: insert just before the 
fileno call
if not sys.__stdout__: raise ValueError()
This is what os.get_terminal_size() raises on IDLE.  Comments in the code in 
posixpath.c make it clear that this is intended guis and the low_level os 
function.

This came up today on Stackoverflow when someone tried to use 
matplotlib.pyplot, which calls os.get_terminal_size, on IDLE.
https://stackoverflow.com/questions/46921087/pyplot-with-idle
(Patching shutil will not help for the os call.)

--
nosy: +terry.reedy
stage:  -> test needed
type: crash -> behavior
versions: +Python 3.7 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Determine the container class of an object in Python 3

[qrious]

Class1 is instantiated in Class2 as follows. Class2 also contains another 
variable, say:

class Class2: 
class1 = Class1()
a = 0

I want to create a method myDef() in Class1 that can read or write to a. How do 
I access a from within myDef() to access a?

Calling Class2.a is not an option as Class1 does not have any knowledge about 
its container class a priori. Also, this will hardcode the path. I want to know 
of a built-in method (kind of equivalent of super() for inheritance) that will 
give me the container class reference, Class2 in this case. 

Thanks. 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:


New changeset 32318930da70ff03320ec50813b843e7db6fbc2e by Alexander Belopolsky 
(Mario Corchero) in branch 'master':
Closes bpo-31800: Support for colon when parsing time offsets (#4015)
https://github.com/python/cpython/commit/32318930da70ff03320ec50813b843e7db6fbc2e


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()

[Guido van Rossum]

Guido van Rossum  added the comment:

SGTM.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

I am going to merge PR 4015 submitted for issue 31800.  That issue asks to make 
%z strptime format accept offsets with : separators.  Given that a similar 
feature has been added to glibc several years ago, I don't see much need for 
bikeshedding.  This issue will remain open and will focus on strftime.

--
dependencies: +datetime.strptime: Support for parsing offsets with a colon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29933] asyncio: set_write_buffer_limits() doc doesn't specify unit of the parameters

[Berker Peksag]

Berker Peksag  added the comment:

PR 2262 has been merged and backported to bugfix branches so I think this issue 
can be closed now.

--
nosy: +berker.peksag
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Berker Peksag]

Change by Berker Peksag :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31702] Allow to specify the number of rounds for SHA-* hashing in crypt

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I'd stick with ValueError in that case as well.  if someone dislikes the
valueerrors because they _want_ to use an invalid one, they can file a bug
and we'll reconsider only if they have a meaningful use case.

On Wed, Oct 25, 2017 at 9:36 AM Serhiy Storchaka 
wrote:

>
> Serhiy Storchaka  added the comment:
>
> What to do with values outside of the valid range (2**4 to 2**31 for
> Blowfish, 1000 to 9 for SHA*). Raise ValueError, OverflowError, or
> bound it, or just generate an invalid salt and allow crypt() to handle it?
>
> --
> nosy: +haypo, pitrou
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31852] Crashes with lines of the form "async \"

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +4091
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18835] Add aligned memory variants to the suite of PyMem functions/macros

[Nathaniel Smith]

Nathaniel Smith  added the comment:

> I'm not sure that it's a good idea to provide a "aligned malloc" fallback if 
> such fallback would be inefficient. For example, we would have to 
> overallocate the memory block not only for the requested alignement, but also 
> allocates extra sizeof(size_t) bytes, *in each* aligned memmory block, to 
> store the size of the alignment itself, to recover the original pointer... to 
> finally be able to call free().

You can re-use the same bytes for padding and to store the offset. The main 
tricky thing is that for an alignment of N bytes you need to overallocate N 
bytes instead of (N-1). (In the worst case, malloc returns you a pointer that's 
already N-byte aligned, and then you have to advance it by a full N bytes so 
that you have some space before the pointer to store the offset.)

Also you want to do a little N = max(N, sizeof(whatever int type you use)) at 
the beginning to make sure it's always big enough, but this is trivial (and 
really even a uint16_t is plenty big to store all reasonable alignments). And 
make sure that N is a power-of-two, which guarantees that whatever value malloc 
returns will be shifted by at least malloc's regular alignment, which is 
guaranteed to be large enough to store a standard int type (on reasonable 
systems).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 7c622be4f2e86c1999baba4f64671a5987d43d73 by Serhiy Storchaka in 
branch '2.7':
bpo-30855: Fix winfo_id related Tkinter test on Windows. (#4121)
https://github.com/python/cpython/commit/7c622be4f2e86c1999baba4f64671a5987d43d73


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Compression of random binary data

[danceswithnumbers]

So if the theoretical min compression limit (log2(n)*(x)) has a 3% margin but 
your transform has a less than 3% inflate rate at most then there is room for 
the transform to compress below the theoretical min. With every transform the 
entropy changes, the potential for greater compression also changes, with each 
pass you can compress untill the entropy is so random it can no longer be 
comressed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Let's talk about debuggers!

[Thomas Jollans]

On 25/10/17 22:18, Terry Reedy wrote:
> On 10/25/2017 12:12 PM, Thomas Jollans wrote:
>> On 2017-10-25 15:57, Rustom Mody wrote:
>>>
>>> pdb inside emacs works (to a fashion)
>>> And it shows the arrow for current line so its at least quasi-gui
>>>
>>> I believe idle too is much more usable than a few years earlier
>>
>> I haven't used IDLE in years (if ever), partly because Tkinter is so
>> incredibly ugly, and feels so out-of-place on a modern PC (maybe it's
>> tolerable on Windows, I wouldn't know).
> 
> On Windows, it uses native widgets when possible, so the look and feel
> of IDLE's editor is the same as some other apps, such as Notepad++.  We
> are in the process of switching to the ttk widget set, which improve the
> look on Linux and even more so on Macs.  The settings dialog was redone,
> in part, for 3.6.3 and 3.7.  Which version are you using?

This was 3.6.0. The look is "meh", but the file selection dialog is (for
me) the real usability disaster.

> 
>> In any case I just tried it out and good lord that's terrible.
> 
> The debugger UI has not gotten its badly needed makeover yet; there is a
> issue and patch to do so.

Glad to hear it!

> 
>> You can
>> set breakpoints in the editor (good), it shows locals (excellent), but
>> it doesn't show you what line you're at when stepping through
>> (seriously?).
> 
> Seriously, you either did not look, or you have a broken IDLE
> installation, or you ran into a severe bug I would like to know about.
> Rather than never, the next line is shown 2 or 3 times.

Sorry Terry, of course you're right. What I meant to write was that it
doesn't *point* to the current line (in the editor) - as apparently
emacs does. I did notice it quoting the current line in the debugger window.

> 
> 1.The module, line number, and code of the next line is shown
> immediately below the header of buttons and checkboxes.
> 
> 2. The box below shows the complete stack of Python lines in the process
> of being run, including (again) the one that will be executed next.  For
> each, it shows the module, line number, and text of the line.
> 
> Bug: code and line number are reversed on the last line, which is the
> repeat of the next line.  Enhancement: make list more readable with
> module, number, and code in neat columns.  The window will have to be
> much wider, or each line split as in tracebacks, to do this.

This sounds like it'll make a nice improvement.

> 
> 3. If one checks the '[X] source' box at the top, the 'next' line is
> highlighted in an IDLE editor window.  This is currently not checked by
> default because the debugger would currently open *every* file stepped
> into.  When opening new windows is made optional, source will be checked
> by default.

Aha!

> 
> In summary, I think debugger should rate at least 'good' rather than
> 'fail' when it comes to showing you the next line.


> 
>> I'll go back to not even touching IDLE with a ten-foot pole now.
> 
> Do as you want, but please report accurately.  There are bugs around the
> debugger, but not, that I know of, the one you claim.

Guilty as charged.


Thomas


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In Python 2 hex() adds the 'L' suffix for longs.

--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4090
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18835] Add aligned memory variants to the suite of PyMem functions/macros

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Android has both memalign() [1] and posix_memalign() [2] and does not have 
aligned_alloc(), posix_memalign() is a wrapper around memalign() [3].

[1] 
https://android.googlesource.com/platform/bionic/+/master/libc/include/malloc.h#38
[2] 
https://android.googlesource.com/platform/bionic/+/master/libc/include/stdlib.h#80
[3] https://android.googlesource.com/platform/bionic/+/85aad90%5E%21/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[m]

W dniu 25.10.2017 o 15:53, Ned Batchelder pisze:
> On 10/25/17 9:07 AM, Thomas Jollans wrote:
>> Hi,
>>
>> I just wanted to know what tools everyone used for debugging Python
>> applications - scripts / backend / desktop apps / notebooks / whatever.
>> Apart from the usual dance with log files and strategically inserted
>> print() calls, that is.
>>
>> Of course we all know and mildly dislike pdb.
>>
>> Personally, in practice, I'm most likely to need a debugger when
>> prototyping a function in a Jupyter notebook. There, ipdb, summoned with
>> the %%debug magic incantation, does the trick.
>>
>> Sometimes, though, I miss having a visual debugger. You know, the kind
>> that Visual Basic has had for decades. There's one in Chrome dev tools
>> if you ever have the misfortune of writing JavaScript.
>>
>> What options are there for Python (that work)? What text editors (and
>> IDEs) have a decent integrated debugger or debugging plugin? (Is there
>> anything for Sublime?) Does anyone use them? (How do YOU debug?)
>>
>> I vaguely remember WinPDB, but that hasn't seen a release in more than
>> seven years...
>>
>>
> 
> pudb is a visual terminal debugger: https://pypi.python.org/pypi/pudb
> 
> It uses the same commands as pdb, so it's easy to get started, but it 
> gives you a variables pane, with customizable presentation, and so on.
> 
> One of my favorite features: you can add a set_trace line in your 
> program, and then if during the debugging session you realize you don't 
> want to stop there every time, you can disable that breakpoint even 
> though it's an explicit line of code telling the debugger to stop.

+1

It's excellent piece of software. I started using it when debugging
remote/server things, and ended with using eveywhere.

It has two drawbacks: it doesn't remember your commandline, watches and
sizes of panes, and has ugly default colour scheme.

Few years ago, I used Eclipse+PyDev, but I was tired with recurring
strange problems with Eclipse, which eg hanged until I reinstalled
eclipse or removed workspace metadata.

r. m.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Let's talk about debuggers!

[Terry Reedy]

On 10/25/2017 12:12 PM, Thomas Jollans wrote:

On 2017-10-25 15:57, Rustom Mody wrote:


pdb inside emacs works (to a fashion)
And it shows the arrow for current line so its at least quasi-gui

I believe idle too is much more usable than a few years earlier


I haven't used IDLE in years (if ever), partly because Tkinter is so
incredibly ugly, and feels so out-of-place on a modern PC (maybe it's
tolerable on Windows, I wouldn't know).


On Windows, it uses native widgets when possible, so the look and feel 
of IDLE's editor is the same as some other apps, such as Notepad++.  We 
are in the process of switching to the ttk widget set, which improve the 
look on Linux and even more so on Macs.  The settings dialog was redone, 
in part, for 3.6.3 and 3.7.  Which version are you using?



In any case I just tried it out and good lord that's terrible.


The debugger UI has not gotten its badly needed makeover yet; there is a 
issue and patch to do so.



You can
set breakpoints in the editor (good), it shows locals (excellent), but
it doesn't show you what line you're at when stepping through (seriously?).


Seriously, you either did not look, or you have a broken IDLE 
installation, or you ran into a severe bug I would like to know about. 
Rather than never, the next line is shown 2 or 3 times.


1.The module, line number, and code of the next line is shown 
immediately below the header of buttons and checkboxes.


2. The box below shows the complete stack of Python lines in the process 
of being run, including (again) the one that will be executed next.  For 
each, it shows the module, line number, and text of the line.


Bug: code and line number are reversed on the last line, which is the 
repeat of the next line.  Enhancement: make list more readable with 
module, number, and code in neat columns.  The window will have to be 
much wider, or each line split as in tracebacks, to do this.


3. If one checks the '[X] source' box at the top, the 'next' line is 
highlighted in an IDLE editor window.  This is currently not checked by 
default because the debugger would currently open *every* file stepped 
into.  When opening new windows is made optional, source will be checked 
by default.


In summary, I think debugger should rate at least 'good' rather than 
'fail' when it comes to showing you the next line.



I'll go back to not even touching IDLE with a ten-foot pole now.


Do as you want, but please report accurately.  There are bugs around the 
debugger, but not, that I know of, the one you claim.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

[issue17942] IDLE Debugger: Improve GUI

[Terry J. Reedy]

Terry J. Reedy  added the comment:

IDLE now uses ttk unconditionally by requiring tk 8.5.  The patch can therefore 
be simplified accordingly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15335] IDLE - debugger steps into print and over rpc.py code

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The debugger also steps into importlib, if one steps 'into' an import 
statement.  Most of the time, this is a nuisance.  If one is importing from 
one's own module, stepping over 'import mymod' is not a satisfactory way to 
avoid this.  Perhaps we should add '(X) Step past IDLE internals' and '(X) Step 
past Python internals', both on by default.  'python internals' would be 
builtin features that happen be implemented, at least in CPython, in python 
rather than the internal language. Other ideas: a drop down skip list with 
items checked.  Or a general policy of skipping Lib/* with a list of exceptions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Tal Einat]

Tal Einat  added the comment:

I understand that there is a workaround. I'm just thinking about the many 
existing large codebases where re-writing thousands of imports because of this 
is unlikely to be done, yet having somewhat longer process launch times would 
be surprising and unwanted.

Anyways I do think it's a very small price to pay for better error messages, 
and there's a good chance nobody will actually feel the difference, so let's 
definitely move forward with this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Brett Cannon]

Brett Cannon  added the comment:

As Nick said, if the overhead of an import statement is that critical, then you 
should NOT use the `from ... import ...` form at all and just stick with 
`import ...` and if necessary, bind local names to objects off of the final 
module or a local name for the overall module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

I would like to add my voice to MAL's objections.  I was not aware of 
time.clock() depreciation before Victor brought this issue to my attention.  
time.clock() is a very well-known API eponymous to a venerable UNIX system 
call.  Its limitations and platform dependencies are very well-known as well.  
For someone new to python, seeing time.clock() would probably not require a 
trip to the library reference, but seeing time.perf_counter() is likely to 
cause a WTF reaction even from an experienced python developer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18835] Add aligned memory variants to the suite of PyMem functions/macros

[Stefan Krah]

Stefan Krah  added the comment:

> In Python 3.7, should we also add the "aligned alloc" requirement?

Linux, BSD, OSX, MSVC should be covered. According to Stackoverflow
MinGW has an internal function.

Android, I don't know. Xavier?

--
nosy: +xdegaye

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[Daniele Forghieri]

Il 25/10/2017 15:07, Thomas Jollans ha scritto:

Hi,

I just wanted to know what tools everyone used for debugging Python
applications - scripts / backend / desktop apps / notebooks / whatever.
Apart from the usual dance with log files and strategically inserted
print() calls, that is.

Of course we all know and mildly dislike pdb.

Personally, in practice, I'm most likely to need a debugger when
prototyping a function in a Jupyter notebook. There, ipdb, summoned with
the %%debug magic incantation, does the trick.

Sometimes, though, I miss having a visual debugger. You know, the kind
that Visual Basic has had for decades. There's one in Chrome dev tools
if you ever have the misfortune of writing JavaScript.

What options are there for Python (that work)? What text editors (and
IDEs) have a decent integrated debugger or debugging plugin? (Is there
anything for Sublime?) Does anyone use them? (How do YOU debug?)

I vaguely remember WinPDB, but that hasn't seen a release in more than
seven years...


I use the PyDev extension to Eclipse IDE, with the standard one at work 
(directly as a plugin to Eclipse) and with LiCplise (a bundle with 
eclipse, pydev and some other plugins) at home.


You can put breakpoint in the code, see all the variables and so forth.

    Daniele Forghieri

--
https://mail.python.org/mailman/listinfo/python-list

[issue31702] Allow to specify the number of rounds for SHA-* hashing in crypt

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What to do with values outside of the valid range (2**4 to 2**31 for Blowfish, 
1000 to 9 for SHA*). Raise ValueError, OverflowError, or bound it, or 
just generate an invalid salt and allow crypt() to handle it?

--
nosy: +haypo, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28564] shutil.rmtree is inefficient due to listdir() instead of scandir()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Following Antoine's suggestion the patch now makes shutil.rmtree() using 
os.scandir() on all platforms.

I doubt about one thing. This patch changes os.listdir passed to the onerror 
handler to os.scandir. This can break a user code that checks if the first 
argument in onerror is os.listdir. If keep this change, it should be documented 
in the "Porting to 3.7" section. Alternatively, we can continue passing 
os.listdir if os.scandir() failed despites the fact that os.listdir no longer 
used.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[Tim]

On Wednesday, October 25, 2017 at 9:07:47 AM UTC-4, Thomas Jollans wrote:
> Hi,
> 
> I just wanted to know what tools everyone used for debugging Python
> applications - scripts / backend / desktop apps / notebooks / whatever.
> Apart from the usual dance with log files and strategically inserted
> print() calls, that is.
> 
> Thomas Jollans

One more vote for pudb. Somehow it fits my brain better than most.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue28564] shutil.rmtree is inefficient due to listdir() instead of scandir()

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think we should change to os.scandir.  No need to accumulate compatibility 
baggage like that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[Thomas Jollans]

On 2017-10-25 15:57, Rustom Mody wrote:
> 
> pdb inside emacs works (to a fashion)
> And it shows the arrow for current line so its at least quasi-gui
> 
> I believe idle too is much more usable than a few years earlier

I haven't used IDLE in years (if ever), partly because Tkinter is so
incredibly ugly, and feels so out-of-place on a modern PC (maybe it's
tolerable on Windows, I wouldn't know).

In any case I just tried it out and good lord that's terrible. You can
set breakpoints in the editor (good), it shows locals (excellent), but
it doesn't show you what line you're at when stepping through (seriously?).

I'll go back to not even touching IDLE with a ten-foot pole now.


-- 
Thomas Jollans
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31803] time.clock() should emit a DeprecationWarning

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This is the current behavior. Do you suggest to undeprecate time.clock() after 
4 releases of deprecation?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

In this case it would be safe to not erase bytes at all.

PR 4119 is too complex for a bugfix (especially if backport it to 3.5 and 3.4). 
It can introduce other regressions. The performance hit is not the only issue. 
Allocating a temporary buffer can change the structure of "holes" in memory. As 
result some memory related bugs can be reproducible only in release mode.

Maybe it is enough to erase only few bytes at the start and end of the freed 
area. The copy can be saved in local variables, without involving the heap. 
This solution still will be enough complex, and I think it can be applied only 
to 3.7. But the bug should be fixed in all affected versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 25.10.2017 01:31, STINNER Victor wrote:
> 
> Marc-Andre: "Yes, to avoid yet another Python 2/3 difference. It should be 
> replaced with the appropriate variant on Windows and non-Windows platforms. 
> From Serhiy's response that's time.process_time() on non-Windows platforms 
> and time.perf_counter() on Windows."
> 
> I don't understand why you mean by "replaced with". Do you mean modify the 
> implementation of the time.clock()?

What I meant is that time.clock() is replaced with the higher
accuracy timers corresponding to the current time.clock()
implementation on the various platforms in order to retain
backwards compatibility.

In other words:

if sys.platform == 'win32':
time.clock = time.perf_counter
else:
time.clock = time.process_time

I know that time.clock() behaves differently on different platforms,
but this fact has been known for a long time and is being used by
Python code out there for timing purposes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31869] commentary on ssl.PROTOCOL_TLS

[J Sloot]

New submission from J Sloot :

on https://docs.python.org/2/library/ssl.html#ssl.PROTOCOL_TLS
for ssl.PROTOCOL_TLS

When the commentary from ssl.PRTOCOL_TLS moved from PROTOCOL_SSLv23 to 
PROTOCOL_TLS the note {Despite the name, this option can select "TLS" protocols 
as well as "SSL"} didn't change.

Suggested new wording {Despite the name, this option can select "SSL" protocols 
as well as "TLS"}

--
assignee: docs@python
components: Documentation
messages: 304996
nosy: J Sloot, docs@python
priority: normal
severity: normal
status: open
title: commentary on ssl.PROTOCOL_TLS
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31852] Crashes with lines of the form "async \"

[STINNER Victor]

STINNER Victor  added the comment:

> Does it make sense to backport ac317700ce7439e38a8b420218d9a5035bba92ed to 
> 3.6?

No, async was not a keyword in Python 3.6 on purpose. Making it a keyword can 
break a lot of code.

I confirm that Python 3.6 still crashs with a very high number of "async " 
prefixes: try attached async_parser_crash.py.

Extract of the gdb traceback on a crash:

(...)
#665 0x00454867 in tok_get (tok=0x7f8b98c0, 
p_start=0x7f8b9cb8, p_end=0x7f8b9cb0) at Parser/tokenizer.c:1571
#666 0x00454867 in tok_get (tok=0x7f8b9d40, 
p_start=0x7f8ba138, p_end=0x7f8ba130) at Parser/tokenizer.c:1571
#667 0x00454867 in tok_get (tok=0x7f8ba1c0, 
p_start=0x7f8ba5b8, p_end=0x7f8ba5b0) at Parser/tokenizer.c:1571
#668 0x00454867 in tok_get (tok=0x7f8ba640, 
p_start=0x7f8baa38, p_end=0x7f8baa30) at Parser/tokenizer.c:1571
#669 0x00454867 in tok_get (tok=0x7f8baac0, 
p_start=0x7f8baeb8, p_end=0x7f8baeb0) at Parser/tokenizer.c:1571
#670 0x00454867 in tok_get (tok=0x7f8baf40, 
p_start=0x7f8bb338, p_end=0x7f8bb330) at Parser/tokenizer.c:1571
#671 0x00454867 in tok_get (tok=0x7f8bb3c0, 
p_start=0x7f8bb7b8, p_end=0x7f8bb7b0) at Parser/tokenizer.c:1571
(...)

It looks like a stack overflow.

The tokenizer may fail earlier on "async async ".

--
nosy: +haypo
Added file: https://bugs.python.org/file47237/async_parser_crash.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Test Bank for Campbell Biology 11th Edition by Urry, Cain

[kkgsrkmk]

Where it didnt come
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31852] Crashes with lines of the form "async \"

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[Petr Viktorin]

Petr Viktorin  added the comment:

tzickel, could you sign the CLA? It will formally allow PSF to distribute your 
patch with Python. (You retain the copyright.) See the dev guide for 
instructions: https://docs.python.org/devguide/pullrequest.html#cla

I can't do that for you, but if you want, I can do the rest (open a pull 
request and convince some core developer to merge it) after you sign the CLA.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[William Ray Wing]

> On Oct 25, 2017, at 9:07 AM, Thomas Jollans  wrote:
> 
> 

[byte]

> What options are there for Python (that work)? What text editors (and
> IDEs) have a decent integrated debugger or debugging plugin?

I rather like WingIDE (the name is a coincidence).  It allows insertion/removal 
of break points while the code is running.  While execution is stopped, it 
allows direct inspection of the stack (no surprise), but in addition allows 
execution of python statements or program elements typed into an auxiliary 
window - including importing things like matplotlib and plotting the current 
state of data arrays.  Its editor is syntax-aware and highlights accidental 
syntax errors as they are typed.  Lots of other features, those just happen to 
be the ones I use most often.

Bill
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[STINNER Victor]

STINNER Victor  added the comment:

> The current code in not correct on all platforms.

My PR 4119 only changes the behaviour on OpenBSD, but I'm not sure about that. 
Maybe it's simpler to apply this fix on all platforms, as I asked on the PR 
itself?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[STINNER Victor]

STINNER Victor  added the comment:

> PR 3844 restores the behavior of 2.7 and 3.3. I propose to merge it first, 
> and develop other enhancements later.

Please don't. This PR reintroduced a bug that I fixed in bpo-18408:

commit c4266360fc70745d49b09f2c29cda91c1a007525
Author: Victor Stinner 
Date:   Tue Jul 9 00:44:43 2013 +0200

Issue #18408: Fix _PyMem_DebugRealloc()

Don't mark old extra memory dead before calling realloc(). realloc() can 
fail
and realloc() must not touch the original buffer on failure.

So mark old extra memory dead only on success if the new buffer did not move
(has the same address).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The current code in not correct on all platforms. We don't know how many of 
random bugs, hangs and crashes on other platforms are caused by this bug. I'm 
not surprised that it was caught on OpenBSD since the robustness and security 
of software is the goal of OpenBSD.

PR 3844 restores the behavior of 2.7 and 3.3. I propose to merge it first, and 
develop other enhancements later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Compression of random binary data

[danceswithnumbers]

Whatever you do, you'll find that *on average* you 
will need *at least* 34 bits to be able to represent 
all possible 10-digit decimal numbers. Some might 
be shorter, but then others will be longer, and 
the average won't be less than 34. 


The theoretical limit for arbitrary numbers 0 - 9 must be viewed as an average 
not as an impossibility to, in some cases be able to compress to or under 34. 
This is obvious by the decimal to binary function.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Let's talk about debuggers!

[Bill]

Fabien wrote:

On 10/25/2017 03:07 PM, Thomas Jollans wrote:

What options are there for Python (that work)?


PyCharm's debugger is fine (also available in the community edition)

+1



Cheers,

Fabien


--
https://mail.python.org/mailman/listinfo/python-list

[issue31664] Add support of new crypt methods

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Tests on s390x SLES 3.x are passed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18835] Add aligned memory variants to the suite of PyMem functions/macros

[STINNER Victor]

STINNER Victor  added the comment:

Currently, the main question on my PR 4089 was raised by Antoine Pitrou:
"Do people have to provide aligned_alloc and aligned_free? Or can they leave 
those pointers NULL and get a default implementation?"

My reply: "Currently, you must provide all allocator functions, included 
aligned_alloc and aligned_free. Technically, we can implement a fallback, but 
I'm not sure that I want to do that :-)"

I'm not sure about that. I can imagine platforms which provide a special 
malloc/free and that's all: no calloc, posix_memalign or _aligned_malloc(). But 
is Python suppose to fills the holes? For example, implement calloc() as 
malloc()+memset()? Or is the user of the PyMem_SetAllocator() API responsible 
to reimplement them?

In Python 3.5, we added the requirement of a working calloc().

In Python 3.7, should we also add the "aligned alloc" requirement?

In case of doubt, I prefer not to guess, and leave the decision to the caller 
of the API: require all functions to be implemented.

I'm not sure that it's a good idea to provide a "aligned malloc" fallback if 
such fallback would be inefficient. For example, we would have to overallocate 
the memory block not only for the requested alignement, but also allocates 
extra sizeof(size_t) bytes, *in each* aligned memmory block, to store the size 
of the alignment itself, to recover the original pointer... to finally be able 
to call free().

An aligned memory block would look like: [A SSS DDD...DDD] where A 
are bytes lost for alignment, SSS bytes storing the alignment size (size of 
"A" in this example), and "DDD...DDD" would be the actual data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[STINNER Victor]

STINNER Victor  added the comment:

"I tried to build CPython on 64-bit OpenBSD. It was built successfully, but 
tests crash."

It's the first time that anyone complains about _PyMem_DebugRawRealloc(). The 
behaviour seems to be very specific to OpenBSD.

Even if the current code is "weird", it works very well and is effecient.

I proposed attached PR 4119 which makes _PyMem_DebugRawRealloc() behaves 
correctly: erase bytes *before* calling realloc(), but keeps a copy if 
realloc() fails. My PR only changes the behaviour on OpenBSD. It keeps the 
current behaviour (erase bytes *after* realloc, if realloc succeeded) on other 
platforms for performance reasons.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +4089

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[Michele Simionato]

pdb plus plus: https://pypi.python.org/pypi/pdbpp
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Let's talk about debuggers!

[Ned Batchelder]

On 10/25/17 9:07 AM, Thomas Jollans wrote:

Hi,

I just wanted to know what tools everyone used for debugging Python
applications - scripts / backend / desktop apps / notebooks / whatever.
Apart from the usual dance with log files and strategically inserted
print() calls, that is.

Of course we all know and mildly dislike pdb.

Personally, in practice, I'm most likely to need a debugger when
prototyping a function in a Jupyter notebook. There, ipdb, summoned with
the %%debug magic incantation, does the trick.

Sometimes, though, I miss having a visual debugger. You know, the kind
that Visual Basic has had for decades. There's one in Chrome dev tools
if you ever have the misfortune of writing JavaScript.

What options are there for Python (that work)? What text editors (and
IDEs) have a decent integrated debugger or debugging plugin? (Is there
anything for Sublime?) Does anyone use them? (How do YOU debug?)

I vaguely remember WinPDB, but that hasn't seen a release in more than
seven years...




pudb is a visual terminal debugger: https://pypi.python.org/pypi/pudb

It uses the same commands as pdb, so it's easy to get started, but it 
gives you a variables pane, with customizable presentation, and so on.


One of my favorite features: you can add a set_trace line in your 
program, and then if during the debugging session you realize you don't 
want to stop there every time, you can disable that breakpoint even 
though it's an explicit line of code telling the debugger to stop.


--Ned.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Let's talk about debuggers!

[Rustom Mody]

On Wednesday, October 25, 2017 at 6:37:47 PM UTC+5:30, Thomas Jollans wrote:
> Hi,
> 
> I just wanted to know what tools everyone used for debugging Python
> applications - scripts / backend / desktop apps / notebooks / whatever.
> Apart from the usual dance with log files and strategically inserted
> print() calls, that is.
> 
> Of course we all know and mildly dislike pdb.
> 
> Personally, in practice, I'm most likely to need a debugger when
> prototyping a function in a Jupyter notebook. There, ipdb, summoned with
> the %%debug magic incantation, does the trick.
> 
> Sometimes, though, I miss having a visual debugger. You know, the kind
> that Visual Basic has had for decades. There's one in Chrome dev tools
> if you ever have the misfortune of writing JavaScript.
> 
> What options are there for Python (that work)? What text editors (and
> IDEs) have a decent integrated debugger or debugging plugin? (Is there
> anything for Sublime?) Does anyone use them? (How do YOU debug?)
> 
> I vaguely remember WinPDB, but that hasn't seen a release in more than
> seven years...

pdb inside emacs works (to a fashion)
And it shows the arrow for current line so its at least quasi-gui

I believe idle too is much more usable than a few years earlier
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: h5py.File() gives error message

[C W]

wow, thanks so much! I don't know how you figured that it's HTML, but
that's awesome!

Mike

On Wed, Oct 25, 2017 at 5:20 AM, Peter Otten <__pete...@web.de> wrote:

> C W wrote:
>
> > Oh, I was running a debug file, that's why the path is different.
> >
> > The file is here,
> > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0
> >
> > Is anyone able to get it working? Thank you!
>
> Hm, that file seems to contain HTML and that causes an OSError here, too:
>
> $ head -n3 datasets/train_catvnoncat.h5
>
> 
> 
>
> $ python3
> Python 3.4.3 (default, Nov 17 2016, 01:08:31)
> [GCC 4.8.4] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import h5py
> >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in
> __init__
> fid = make_fid(name, mode, userblock_size, fapl)
>   File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in
> make_fid
> fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl)
>   File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806)
> OSError: unable to open file (File accessibilty: Unable to open file)
>
> It's not exactly what you see, but that may be due to differing software
> versions.
> When I replace the HTML file with its namesake found at
>
> https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/cod
> e/datasets/train_catvnoncat.h5
>
> I can open it:
>
> $ file datasets/train_catvnoncat.h5
> datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data
>
> $ python3
> Python 3.4.3 (default, Nov 17 2016, 01:08:31)
> [GCC 4.8.4] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import h5py
> >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")
> >>>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31626] Writing in freed memory in _PyMem_DebugRawRealloc() after shrinking a memory block

[STINNER Victor]

STINNER Victor  added the comment:

> Using nested _PyMem_DebugRawRealloc() looks suspicions to me. This may be a 
> bug.

PyObject_Malloc() calls PyMem_RawMalloc() for allocations larger than 512 bytes.

When debug hooks are enabled, PyObject_Malloc() and PyMem_RawMalloc() both call 
_PyMem_DebugRawRealloc(). The behaviour that you saw is expected.

It was simpler to reuse _PyMem_DebugRawRealloc() PyObject and PyMem_Raw 
allocator families, rather than duplicating the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Tal Einat]

Tal Einat  added the comment:

I can't say I agree that the performance here is practically insignificant. 
This will affect the startup time of Python process, and adding even 10% to 
that in some cases is significant.

In some of the larger codebases I've worked on, even simple scripts would 
import large portions of the system, and there would be thousands of such 
imports done in the process. There are "basic" utilities in the stdlib which 
are imported very often in different modules, so the performance of the import 
statement is not necessarily insignificant compared to that of actually loading 
the modules.

That being said I'm all for getting this in and implementing an optimization of 
the slower path in time for 3.7.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Let's talk about debuggers!

[Fabien]

On 10/25/2017 03:07 PM, Thomas Jollans wrote:

What options are there for Python (that work)?


PyCharm's debugger is fine (also available in the community edition)

Cheers,

Fabien
--
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

[Ian Kelly]

On 10/24/17, Richard Damon  wrote:
> My understanding of the 'Random Data Comprehensibility' challenge is
> that is requires that the compression take ANY/ALL strings of up to N
> bits, and generate an output stream no longer than the input stream, and
> sometime less.

That's incorrect, at least of the challenge being discussed. Here's the link:

http://marknelson.us/2012/10/09/the-random-compression-challenge-turns-ten/

The challenge is just to take a single known file of a million random
digits and make it smaller, including the size of the decompresser and
without hiding data. So far in 15 years nobody has succeeded even at
this, and nobody knows whether it's impossible. For instance it may be
the case that the data in the file happens to be the nth prime, in
which case it could simply be compressed to the number n with a
decompresser that calculates process.

> It admits that given no-uniformly distributed data, it is
> possible to compress some patterns, the common ones, and expand others,
> the uncommon ones, to lower the net average length. What it says can't
> be done is to have a compression method that compresses EVERY input
> pattern. That is where the 'Pigeon Hole' principle comes into play which
> the people who claim to be able to compress random data like to ignore
> or just attempt to say doesn't apply.

There is a second challenge on that page that is as you state, but the
page admits that this second challenge is a bit of a troll since this
is provably impossible.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue31664] Add support of new crypt methods

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 0f261583bae7e60e410709ed96398dd1b14c5454 by Serhiy Storchaka in 
branch 'master':
bpo-31664: Fix test_crypt for the openwall implementation of crypt. (#4116)
https://github.com/python/cpython/commit/0f261583bae7e60e410709ed96398dd1b14c5454


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Let's talk about debuggers!

[Thomas Jollans]

Hi,

I just wanted to know what tools everyone used for debugging Python
applications - scripts / backend / desktop apps / notebooks / whatever.
Apart from the usual dance with log files and strategically inserted
print() calls, that is.

Of course we all know and mildly dislike pdb.

Personally, in practice, I'm most likely to need a debugger when
prototyping a function in a Jupyter notebook. There, ipdb, summoned with
the %%debug magic incantation, does the trick.

Sometimes, though, I miss having a visual debugger. You know, the kind
that Visual Basic has had for decades. There's one in Chrome dev tools
if you ever have the misfortune of writing JavaScript.

What options are there for Python (that work)? What text editors (and
IDEs) have a decent integrated debugger or debugging plugin? (Is there
anything for Sublime?) Does anyone use them? (How do YOU debug?)

I vaguely remember WinPDB, but that hasn't seen a release in more than
seven years...


-- 
Thomas Jollans
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There are other differences between Python 2.7 and Python 3. PR 4118 restores 
the Python 2.7 logic. It adds type checking, but its overhead is smaller.

$ ./python -m timeit 'from encodings import aliases'
50 loops, best of 5: 542 nsec per loop

Actually in some cases (with '*') the new code is even slightly faster.

I don't know whether these differences were intentional, but all tests are 
passed.

The type of items in __all__ also is checked.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4088

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used

[STINNER Victor]

STINNER Victor  added the comment:

Thank you Anselm Kruis for spotting this nice optimization opportunity! Sadly, 
as I wrote, I don't want to backport the optimization to the stable Python 3.6 
branch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 086c3ae5f0995a62092b9080f32dd118c2923453 by Victor Stinner in 
branch 'master':
bpo-31835: Optimize also FASTCALL using __future__ (#4087)
https://github.com/python/cpython/commit/086c3ae5f0995a62092b9080f32dd118c2923453


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9 files

[Tal Einat]

Tal Einat  added the comment:

See PR 4117 on GitHub with the itertools AC conversion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9 files

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +4087
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

SciPy 1.0 released!

[Ralf Gommers]

Hi all,

We are extremely pleased to announce the release of SciPy 1.0, 16 years
after
version 0.1 saw the light of day.  It has been a long, productive journey to
get here, and we anticipate many more exciting new features and releases in
the
future.


Why 1.0 now?


A version number should reflect the maturity of a project - and SciPy was a
mature and stable library that is heavily used in production settings for a
long time already.  From that perspective, the 1.0 version number is long
overdue.

Some key project goals, both technical (e.g. Windows wheels and continuous
integration) and organisational (a governance structure, code of conduct
and a
roadmap), have been achieved recently.

Many of us are a bit perfectionist, and therefore are reluctant to call
something "1.0" because it may imply that it's "finished" or "we are 100%
happy
with it".  This is normal for many open source projects, however that
doesn't
make it right.  We acknowledge to ourselves that it's not perfect, and there
are some dusty corners left (that will probably always be the case).
Despite
that, SciPy is extremely useful to its users, on average has high quality
code
and documentation, and gives the stability and backwards compatibility
guarantees that a 1.0 label imply.


Some history and perspectives
-

- 2001: the first SciPy release
- 2005: transition to NumPy
- 2007: creation of scikits
- 2008: scipy.spatial module and first Cython code added
- 2010: moving to a 6-monthly release cycle
- 2011: SciPy development moves to GitHub
- 2011: Python 3 support
- 2012: adding a sparse graph module and unified optimization interface
- 2012: removal of scipy.maxentropy
- 2013: continuous integration with TravisCI
- 2015: adding Cython interface for BLAS/LAPACK and a benchmark suite
- 2017: adding a unified C API with scipy.LowLevelCallable; removal of
scipy.weave
- 2017: SciPy 1.0 release


**Pauli Virtanen** is SciPy's Benevolent Dictator For Life (BDFL).  He says:

*Truthfully speaking, we could have released a SciPy 1.0 a long time ago,
so I'm
happy we do it now at long last. The project has a long history, and during
the
years it has matured also as a software project.  I believe it has well
proved
its merit to warrant a version number starting with unity.*

*Since its conception 15+ years ago, SciPy has largely been written by and
for
scientists, to provide a box of basic tools that they need. Over time, the
set
of people active in its development has undergone some rotation, and we have
evolved towards a somewhat more systematic approach to development.
Regardless,
this underlying drive has stayed the same, and I think it will also continue
propelling the project forward in future. This is all good, since not long
after 1.0 comes 1.1.*

**Travis Oliphant** is one of SciPy's creators.  He says:

*I'm honored to write a note of congratulations to the SciPy developers and
the
entire SciPy community for the release of SciPy 1.0.   This release
represents
a dream of many that has been patiently pursued by a stalwart group of
pioneers
for nearly 2 decades.   Efforts have been broad and consistent over that
time
from many hundreds of people.   From initial discussions to efforts coding
and
packaging to documentation efforts to extensive conference and community
building, the SciPy effort has been a global phenomenon that it has been a
privilege to participate in.*

*The idea of SciPy was already in multiple people’s minds in 1997 when I
first
joined the Python community as a young graduate student who had just fallen
in
love with the expressibility and extensibility of Python.   The internet was
just starting to bringing together like-minded mathematicians and
scientists in
nascent electronically-connected communities.   In 1998, there was a
concerted
discussion on the matrix-SIG, python mailing list with people like Paul
Barrett, Joe Harrington, Perry Greenfield, Paul Dubois, Konrad Hinsen, David
Ascher, and others.   This discussion encouraged me in 1998 and 1999 to
procrastinate my PhD and spend a lot of time writing extension modules to
Python that mostly wrapped battle-tested Fortran and C-code making it
available
to the Python user.   This work attracted the help of others like Robert
Kern,
Pearu Peterson and Eric Jones who joined their efforts with mine in 2000 so
that by 2001, the first SciPy release was ready.   This was long before
Github
simplified collaboration and input from others and the "patch" command and
email was how you helped a project improve.*

*Since that time, hundreds of people have spent an enormous amount of time
improving the SciPy library and the community surrounding this library has
dramatically grown. I stopped being able to participate actively in
developing
the SciPy library around 2010.  Fortunately, at that time, Pauli Virtanen
and
Ralf Gommers picked up the pace of development supported by dozens of other
key
contributors such as David Cournapeau, Evgeni 

PyCA cryptography 2.1.2 released

[Paul Kehrer]

PyCA cryptography 2.1.2 has been released to PyPI. cryptography includes
both high level recipes and low level interfaces
to common cryptographic algorithms such as symmetric ciphers, message
digests, and key derivation functions. We support Python 2.7, Python 3.4+,
and PyPy.

Changelog:
* This release corrects a bug with the manylinux1 wheels where OpenSSL's
stack was marked executable.

-Paul Kehrer (reaperhulk)
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

[issue31868] Null pointer dereference in ndb.ndbm get when used with a default value.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Could you please create a pull request on GitHub Tomasz?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31867] Duplicated keys in MIME type_map with different values

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

So for excel, 'application/excel' is not listed at 
http://www.iana.org/assignments/media-types as suggested by the comment at:
https://github.com/python/cpython/blob/master/Lib/mimetypes.py#L397
whereas 'applicaton/vnd.ms-excel' is listed. See also
https://stackoverflow.com/questions/974079/setting-mime-type-for-excel-document.

For cdf, Wolframalpha is using application/cdf it seems (without it being 
registered) whereas NetCDF suggests application/x-netcdf so that seems 
reasonable (and is the current value).

For context, these duplicates were introduced here:
https://github.com/python/cpython/commit/a3689fe78675c89d5979c0c5acdb1c173cc75ed6
and noted first here:
https://github.com/python/cpython/commit/107771a228ee73b4683242cb696e3024f93b74d5

--
nosy: +Henk-Jaap Wagenaar
title: Duplicated keys, but with different values in dictionary literals -> 
Duplicated keys in MIME type_map with different values

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31664] Add support of new crypt methods

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4086
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

[Andy]

Andy  added the comment:

Thanks for adding the test!

If the official stance is that only the latest OpenSSL is supported then this 
is definitely WAI. Sounds like a good policy...

I'll close this issue.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31868] Null pointer dereference in ndb.ndbm get when used with a default value.

[Tomasz Miąsko]

New submission from Tomasz Miąsko :

Using ndb.ndbm get when key is missing and default value has to be returned
results in invocation of Py_INCREF on null pointer. Test case to reproduce the 
issue:

```
import dbm.ndbm

with dbm.ndbm.open('db', 'n') as db:
print(db.get('missing-key'))
```

--
files: 0001-Fix-null-pointer-dereference-in-dbm.ndbm-get.patch
keywords: patch
messages: 304977
nosy: tmiasko
priority: normal
severity: normal
status: open
title: Null pointer dereference in ndb.ndbm get when used with a default value.
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: 
https://bugs.python.org/file47236/0001-Fix-null-pointer-dereference-in-dbm.ndbm-get.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9 files

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Yes, it is. Actually the #include should be before lists of methods and static 
PyTypeObject initializers. I think it is better to move all method lists and 
type initializers to the bottom of the file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31852] Crashes with lines of the form "async \"

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This issue is fixed in the master branch (version 3.7.0 alpha 2). The issue was 
fixed in this PR:

https://github.com/python/cpython/pull/1669

The cause is that async was not a proper keyword and the parser segfaults when 
checking for the new token and parsing the newline. In particular, this happens 
here:

translate_newlines at Parser/tokenizer.c:713
713 buf = PyMem_MALLOC(needed_length);

This is the stack trace:

#0  _PyObject_Alloc (ctx=, elsize=10, nelem=1, use_calloc=0) at 
Objects/obmalloc.c:806
#1  _PyObject_Malloc (ctx=, nbytes=10) at Objects/obmalloc.c:985
#2  0x00453020 in translate_newlines (tok=0x9187b0, exec_input=0, 
s=0x77fa40e0 "async \\\n") at Parser/tokenizer.c:713
#3  tok_nextc (tok=tok@entry=0x9187b0) at Parser/tokenizer.c:943
#4  0x00454948 in tok_get (tok=tok@entry=0x9187b0, 
p_start=p_start@entry=0x7fffdc40, p_end=p_end@entry=0x7fffdc50)
at Parser/tokenizer.c:1382
#5  0x00455749 in PyTokenizer_Get (tok=tok@entry=0x9187b0, 
p_start=p_start@entry=0x7fffdc40, p_end=p_end@entry=0x7fffdc50)
at Parser/tokenizer.c:1902
#6  0x0045158d in parsetok (tok=0x9187b0, g=, start=256, 
err_ret=err_ret@entry=0x7fffdce0,
flags=flags@entry=0x7fffdcd0) at Parser/parsetok.c:208
#7  0x00452280 in PyParser_ParseFileObject (fp=, 
filename=filename@entry=0x77f1b848, enc=,
g=, start=, ps1=, 
ps2=0x77e63648 "... ", err_ret=err_ret@entry=0x7fffdce0,
flags=flags@entry=0x7fffdcd0) at Parser/parsetok.c:134
#8  0x00433949 in PyParser_ASTFromFileObject (fp=, 
filename=0x77f1b848, enc=,
start=, ps1=, ps2=, 
flags=0x7fffde90, errcode=0x7fffdd80, arena=0x77fe2168)
at Python/pythonrun.c:1166
#9  0x00433b5b in PyRun_InteractiveOneObject 
(fp=fp@entry=0x774b2640 <_IO_2_1_stdin_>, 
filename=filename@entry=0x77f1b848,
flags=flags@entry=0x7fffde90) at Python/pythonrun.c:218
#10 0x00433eae in PyRun_InteractiveLoopFlags 
(fp=fp@entry=0x774b2640 <_IO_2_1_stdin_>,
filename_str=filename_str@entry=0x5dd7a4 "", 
flags=flags@entry=0x7fffde90) at Python/pythonrun.c:115
#11 0x00433fbc in PyRun_AnyFileExFlags (fp=0x774b2640 
<_IO_2_1_stdin_>, filename=0x5dd7a4 "", closeit=0,
flags=0x7fffde90) at Python/pythonrun.c:77
#12 0x004476fa in run_file (p_cf=0x7fffde90, filename=, fp=0x774b2640 <_IO_2_1_stdin_>) at Modules/main.c:341
#13 Py_Main (argc=argc@entry=1, argv=argv@entry=0x910010) at Modules/main.c:895
#14 0x0041e17a in main (argc=1, argv=) at 
./Programs/python.c:102

After applying commit ac317700ce7439e38a8b420218d9a5035bba92ed the issue is 
fixed.

Does it make sense to backport ac317700ce7439e38a8b420218d9a5035bba92ed to 3.6?

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31867] Duplicated keys, but with different values in dictionary literals

[Mark Shannon]

Mark Shannon  added the comment:

I hadn't noticed the comments on the lines above saying "Duplicates :(", so I'm 
obviously not the first to notice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31867] Duplicated keys, but with different values in dictionary literals

[Mark Shannon]

New submission from Mark Shannon :

Here
https://github.com/python/cpython/blob/master/Lib/mimetypes.py#L416
and here
https://github.com/python/cpython/blob/master/Lib/mimetypes.py#L526

I have no idea which is the correct value for either, but the code is 
misleading at best.

--
components: Library (Lib)
messages: 304973
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Duplicated keys, but with different values in dictionary literals
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20180] Derby #11: Convert 50 sites to Argument Clinic across 9 files

[Tal Einat]

Tal Einat  added the comment:

As far as I can tell, with the creation of a separate clinic/itertools.c.h, the 
typedefs etc. need to be moved to the top of the file so that the #include can 
come after the typedefs and before the rest of the code which references clinic 
output. Is this indeed the way to go?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: right list for SIGABRT python binary question ?

[Karsten Hilbert]

On Tue, Oct 24, 2017 at 08:47:58PM +0200, M.-A. Lemburg wrote:

> >> This error suggests that you have 32- and 64-bit versions of
> >> Python and mxDateTime mixed in your installation.
> >>
> >> Py_InitModule4 is only available in the 32-bit build of
> >> Python. With the 64-bit build, it's called Py_InitModule4_64.
...
> Could you check whether you have similar import errors with
> other modules that have C extensions ? E.g. lxml.
> 
> What you're seeing appears to be a compilation problem
> with Python 2.7.14 on Debian. The executable doesn't appear
> to export its symbols to the .so files, or only some of them.

Let's see:

python-lxml:
  Installiert:   4.0.0-1
  Installationskandidat: 4.0.0-1
  Versionstabelle:
 *** 4.0.0-1 990
990 http://httpredir.debian.org/debian buster/main i386 Packages
500 http://httpredir.debian.org/debian unstable/main i386 
Packages
100 /var/lib/dpkg/status
 3.7.1-1 500
500 http://httpredir.debian.org/debian stretch/main i386 
Packages
python-lxml-dbg:
  Installiert:   (keine)
  Installationskandidat: 4.0.0-1
  Versionstabelle:
 4.0.0-1 990
990 http://httpredir.debian.org/debian buster/main i386 Packages
500 http://httpredir.debian.org/debian unstable/main i386 
Packages
 3.7.1-1 500
500 http://httpredir.debian.org/debian stretch/main i386 
Packages


ncq@hermes:~$ python2.7-dbg
Python 2.7.14 (default, Sep 17 2017, 18:50:44)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml
[45350 refs]
>>>

ncq@hermes:~$ python
Python 2.7.14 (default, Sep 17 2017, 18:50:44)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml
>>>

Also, psycogp2 imports just fine.

Now that I have

python-egenix-mx-base-dbg:
  Installiert:   3.2.9-1
  Installationskandidat: 3.2.9-1
  Versionstabelle:
 *** 3.2.9-1 990
500 http://httpredir.debian.org/debian stretch/main i386 
Packages
990 http://httpredir.debian.org/debian buster/main i386 Packages
500 http://httpredir.debian.org/debian unstable/main i386 
Packages
100 /var/lib/dpkg/status

mx.DateTime imports fine as well (but the SIGABRT persists).

Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The from import already is much slower than simple import:

$ ./python -m timeit 'from encodings import aliases'
50 loops, best of 5: 475 nsec per loop
$ ./python -m timeit 'import encodings.aliases as aliases'
100 loops, best of 5: 289 nsec per loop

The latter executes only C code if the module already is imported, but the 
former executes Python code. It may be worth to add the C acceleration for this 
case too.

PR 4113 makes it yet slower:
 
$ ./python -m timeit 'from encodings import aliases'
50 loops, best of 5: 793 nsec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: h5py.File() gives error message

[Peter Otten]

C W wrote:

> Oh, I was running a debug file, that's why the path is different.
> 
> The file is here,
> https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0
> 
> Is anyone able to get it working? Thank you!

Hm, that file seems to contain HTML and that causes an OSError here, too:

$ head -n3 datasets/train_catvnoncat.h5 




$ python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py
>>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in __init__
fid = make_fid(name, mode, userblock_size, fapl)
  File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in make_fid
fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl)
  File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806)
OSError: unable to open file (File accessibilty: Unable to open file)

It's not exactly what you see, but that may be due to differing software 
versions.
When I replace the HTML file with its namesake found at

https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/code/datasets/train_catvnoncat.h5

I can open it:

$ file datasets/train_catvnoncat.h5 
datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data

$ python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py
>>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")
>>>

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue24564] shutil.copytree fails when copying NFS to NFS

[Albert Zeyer]

Albert Zeyer  added the comment:

I'm also affected by this, with Python 3.6. My home directory is on a 
ZFS-backed NFS share.
See here for details:
https://github.com/Linuxbrew/homebrew-core/issues/4799

Basically:
Copying setuptools.egg-info to 
/u/zeyer/.linuxbrew/lib/python3.6/site-packages/setuptools-36.5.0-py3.6.egg-info
error: [Errno 5] Input/output error: 
'/u/zeyer/.linuxbrew/lib/python3.6/site-packages/setuptools-36.5.0-py3.6.egg-info/zip-safe'

Note that also by other tools, such as `mv` and `cp`, I get errors about 
setting `system.nfs4_acl`. But they just seem to ignore that and go on. I think 
this is the right thing to do here. You can print a warning about that, but 
then just go on. Maybe esp. just for `system.nfs4_acl`.

--
nosy: +Albert.Zeyer
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Nick Coghlan]

Nick Coghlan  added the comment:

I'm fine with the approach in the latest version of the PR - it does make "from 
x import y" slightly slower due to the extra error checking, but folks should 
be avoiding doing that in performance critical loops or functions anyway.

It would be nice if we could avoid that overhead for the import statement case, 
but we can't readily tell the difference between "__import__ called via syntax" 
and "__import__ called directly", and I don't think this is going to be 
performance critical enough to be worth introducing that complexity.

The question of how best to handle passing a consumable iterator as the from 
list would be a separate question, if we decided to do anything about it at all 
(the current implementation implicitly assumes the input is reiterable, but 
passing a non-container seems like a harder mistake to make than accidentally 
passing bytes instead of a string).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[STINNER Victor]

STINNER Victor  added the comment:

Too bad, the bug is not dead:

http://buildbot.python.org/all/#/builders/45/builds/14

test_use (test_tkinter.test_widgets.ToplevelTest) ... ERROR
test_visual (test_tkinter.test_widgets.ToplevelTest) ... ok
test test_tk failed -- Traceback (most recent call last):
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\test\test_tkinter\test_widgets.py",
 line 92, in test_use
widget2 = self.create(use=wid)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\test\test_tkinter\test_widgets.py",
 line 67, in create
return tkinter.Toplevel(self.root, **kwargs)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\Tkinter.py", 
line 2138, in __init__
BaseWidget.__init__(self, master, 'toplevel', cnf, {}, extra)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\Tkinter.py", 
line 2095, in __init__
(widgetName, self._w) + extra + self._options(cnf))
TclError: expected integer but got "0x8e5c00d6L"
test_width (test_tkinter.test_widgets.ToplevelTest) ... ok
==
ERROR: test_use (test_tkinter.test_widgets.ToplevelTest)
--
Traceback (most recent call last):
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\test\test_tkinter\test_widgets.py",
 line 92, in test_use
widget2 = self.create(use=wid)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\test\test_tkinter\test_widgets.py",
 line 67, in create
return tkinter.Toplevel(self.root, **kwargs)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\Tkinter.py", 
line 2138, in __init__
BaseWidget.__init__(self, master, 'toplevel', cnf, {}, extra)
  File 
"d:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\lib-tk\Tkinter.py", 
line 2095, in __init__
(widgetName, self._w) + extra + self._options(cnf))
TclError: expected integer but got "0x8e5c00d6L"

--
nosy: +terry.reedy
resolution: out of date -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Sockets but calling from different programs

[Cameron Simpson]

On 23Oct2017 05:33, T Obulesu  wrote:

I'm new to python3 and scratching my head to write a program for this logic:


The tutor list might be a better place for such questions, but since we're 
here...



classA.py
Class A:
 # class for socket communication
 basic init method that initializes port, address, connection
method send(message):
# for sending any message through the given port
  method receive():
  # will be keep on listening for incoming messages

classB.py
Class B:
  import the send method from class A
  send the messages from this class

classC.py
Class C:
   import the receive method from the class A
   receive all the messages from the same socket from here.

Note:
classA.py, classB.py, ClassC.py are saved in different locations.

Can someone help me in writing the code and how can I create a single object 
and use it in multiple classed?


That is a pretty normal arrangement. Class A might look like this:

 class A:
   def __init__(self, port, address):
 self.connection = ... make your connection to (address, port)
   def send(self, msg):
 send msg using self.connection ...

Since classes B and C seem expected to share tha same connection, the natural 
thing is to set up the connection _separately_ from setting up B and C, and 
pass the established connection to each.


So class B might commence:

 class B:
   def __init__(self, conn, ...):
 self.conn = conn
 ... whatever other initialisation ...
   def speak(self, something):
 self.conn.send(something)

You'll notice here that we're _not_ importing anything about class A here.  
Class B does not need to know class A's name to use it.


Because Python uses duck typing, you could pass _any_ object which has a 
.send() method as "conn" to the class B setup. This allows you to write some 
other class A2 with those same methods, but using some different type of 
connection, and pass that in to classes B and C.


So a main programme might set things up like this:

 from classA import A
 from classB import B
 from classC import C

 conn = A(address, port)
 sender = B(conn, other-stuff...)
 receiver = C(conn, other-stuff...)
 B.speak("foo")

and so forth.

Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
--
https://mail.python.org/mailman/listinfo/python-list

[issue21720] "TypeError: Item in ``from list'' not a string" message

[Berker Peksag]

Berker Peksag  added the comment:

> I don't think the index in error message is needed.

I'm fine with either format. It's ultimately up to Nick. Should I switch back 
to the 2.7 version?

> Import is successful because the iterator was exhausted by "'*' in fromlist".

This shouldn't be a problem in the latest version of PR 4113. While I don't 
think anyone would pass ``fromlist=iter(('aliases', b'foobar'))`` in real life, 
I can add it to the test. Let me know what do you think.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Compression of random binary data

[Gregory Ewing]

Lele Gaifax wrote:

That's simple enough: of course one empty file would be
"music.mp3.zip.zip.zip", while the other would be
"movie.avi.zip.zip.zip.zip.zip"... some sort of
https://en.wikipedia.org/wiki/Water_memory applied to file system entries :-)


If you're allowed to alternate between two compression
methods, then the way you decompress
music.mp3.zip.zip.tgz.zip...tgz.zip.tgz
is to output 0 each time zip was applied and 1 each
time tar/gz was applied.

You may be able to take some shortcuts in some
cases, e.g. anything beginning with "movie.flv"
almost certainly contains a cute kitten video.
(Unless it's found inside an encrypted disk
partition, in which case it contains porn.)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

[Gregory Ewing]

Ben Bacarisse wrote:

The trouble is a pedagogic one.  Saying "you can't compress random data"
inevitably leads (though, again, this is just my experience) to endless
attempts to define random data.


It's more about using terms without making sure everyone agrees
on the definitions being used.

In this context, "random data" really means "uniformly distributed
data", i.e. any bit sequence is equally likely to be presented as
input. *That's* what information theory says can't be compressed.


I think "arbitrary data" (thereby including the results of compression
by said algorithm) is the best way to make progress.


I'm not sure that's much better, because it doesn't home in
on the most important thing, which is the probability
distribution.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

[issue28503] [Patch] '_crypt' module: fix implicit declaration of crypt(), use crypt_r() where available

[Ed Schouten]

Ed Schouten  added the comment:

Having looked at various implementations of crypt() and crypt_r(), I can't 
think of a reason why there would be any significant difference in performance. 
On systems like FreeBSD, crypt() is just a simple wrapper around crypt_r():

https://svnweb.freebsd.org/base/head/lib/libcrypt/crypt.c?view=markup#l134

If there would be any difference in performance, it would also be 
astronomically small compared to the computation time spent by the 
cryptographic hash functions themselves.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com