Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Antoon Pardon]

Op 27/01/21 om 05:17 schreef Dan Stromberg:

On Tue, Jan 26, 2021 at 8:13 PM Dan Stromberg  wrote:


On Tue, Jan 26, 2021 at 4:01 PM C W  wrote:


Hello everyone,

I'm a long time Matlab and R user working on data science. How do you
troubleshooting/debugging in Python?


I frequently read tracebacks and think about what's up in the code.

I also often add print functions or logging - empiricism often beats
theorizing when the problems are weird.

And once in a while I will use pudb - it's probably a pretty good fit for
a vim user like me, both being curses-based.  pdb is sad.  There are other
debuggers for Python:
https://wiki.python.org/moin/PythonDebuggingTools


BTW, there's a new tool I haven't tried yet, that sounds pretty
interesting, especially for newcomers: It's called "friendly traceback" and
can be found at https://pypi.org/project/friendly-traceback/

Oh, and of course google search is a terrific tool for deciphering error
messages.

I had a look at that and I think I still would prefer this recipe:
https://code.activestate.com/recipes/52215-get-more-information-from-tracebacks/

--
https://mail.python.org/mailman/listinfo/python-list

[issue43048] Printing RecursionError results in RecursionError

[Irit Katriel]

Irit Katriel  added the comment:

This looks like a duplicate of issue 42848 which was fixed in 3.10.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43040] random.py randrange() is very slow if the range is a power of 2.

[Donovan Baarda]

Donovan Baarda  added the comment:

Some points to note;

I first noticed this as an apparently 5x performance regression for randrange() 
between pypy and pypy3. This seemed pretty significant, but admittedly the 
difference is largely masked by other python overheads when comparing python2 
and python3.

When I mentioned random() is faster, I meant that python2's implementation of 
randrange() that uses random() under the hood was noticeably faster than 
python3's randrange() that uses getrandbits() under the hood.

I know getrandbits() is significantly faster than randrange(), but randrange() 
doesn't have to be that bad.

However, I agree that changing repeatability is potentially a big issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Grant Edwards]

On 2021-01-27, C W  wrote:

> My main takeaway from the discussion so far is that: you can't troubleshoot
> Python without some kind of breakpoint or debugger.

How odd. I do it all the time.

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43052] _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0

[Isuru Fernando]

New submission from Isuru Fernando :

In macOS Big Sur, if the executable was compiled with 
MACOSX_DEPLOYMENT_TARGET=10.15
or below, then SYSTEM_VERSION_COMPAT=1 is the default which means that Big Sur
reports itself as 10.16 which in turn means that __builtin_available(macOS 11.0)
will not be triggered.

This can be observed by using the python 3.9.1 universal2 installer and using 
it on
x86_64 Big Sur or with Rossetta 2 on arm64 Big Sur. (Not an issue with native 
arm64
as that part is compiled with MACOSX_DEPLOYMENT_TARGET=11.0)

Original issue is that the following returned None.
SYSTEM_VERSION_COMPAT=1 arch -x86_64 /usr/local/bin/python3 -c "from 
ctypes.util import find_library; print(find_library('AppKit'))"

--
messages: 385845
nosy: isuruf, ned.deily, ronaldoussoren
priority: normal
pull_requests: 23185
severity: normal
status: open
title: _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43025] Use normal 'i' character to denote imaginary part of complex numbers

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Unfortunately, this ship sailed a long time ago.  Changing it now would create 
more problems that it solves.

> I doubt that that will be enough to silence the repeated requests:
> people are still going to complain that the value that was entered
> as `1i` is being displayed as `1j`. I don't see a sane path 
> towards changing that.

Exactly this.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Sounds like another wet blanket argpment

Please watch your tone.  It borders on being abusive.

It is entirely appropriate to set limits on the scope of what a module is 
trying to achieve.  It is appropriate to consider module learnability.  It is 
appropriate to limit sprawl and feature creep.  It is appropriate to consider 
whether the pattern being encouraged and supported by the PR is intellectually 
manageable by end users.

It is correct that argparse set out to handle more complex cases than optparse. 
 But even then, it was aiming to cases that commonly arose it the wild 
(subparsers are not uncommon).  It never aspired to be all things to all users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36086] Split IDLE into separate feature in Windows installer

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I concur with the other commenters that this isn't worth it.  Installing Tcl/Tk 
without IDLE saves almost nothing.

Thank you for the suggestion, but I'm going to mark it as rejected.

--
nosy: +rhettinger
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43034] Python tutorial misleads users about floor division behavior

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The first section of the tutorial isn't the right place to go into these 
details.  The point of the section is to have a basic introduction to the 
interactive prompt.  It is intensionally gentle and lightweight.

It isn't the purpose of the tutorial to document complete semantics and corner 
cases.  We leave that for the language reference or for coverage of specific 
types in the library reference.  See footnote (1) in the operator table at 
https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex 
.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43040] random.py randrange() is very slow if the range is a power of 2.

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43040] random.py randrange() is very slow if the range is a power of 2.

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Donovan, if the speed of the power of two case is important to you, I recommend 
using getrandbits() instead.  It is ten times faster.

  $ python3.9 -m timeit -r 11 -s 'from random import randrange, getrandbits' 
'randrange(2**32)'
  100 loops, best of 11: 362 nsec per loop

  $ python3.9 -m timeit -r 11 -s 'from random import randrange, getrandbits' 
'getrandbits(32)'
  1000 loops, best of 11: 32 nsec per loop

The two referenced articles aren't pertinent to this issue, the power of two 
case for randrange(). Both articles boil down to finding that a single task, 
arity zero, C coded method is significantly faster than a pure python method 
with error checking, a complex signature, support for arbitrarily large 
integers, and extra effort to assure an equal distribution.  It is unsurprising 
random() is faster than randrange().  Code that does less work is faster than 
code that does more work.

Mark is correct that the current code wasn't an accident and that we 
intensionally chose to keep reproducibility.

I appreciate your suggestion, but it essentially same one that was rejected in 
bpo-37000.  As Mark pointed out, nothing has changed since then (though some 
other changes have been added to 3.10 that improved the speed of rangerange() 
without affecting reproducibility).

Note, even if the PR had been accepted, you would still be much better off with 
getrandbits().

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> _randbelow_with_getrandbits function inefficient with powers of 
two

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to accept argparse.log_level parameter and propagate its value to all other modules

[Zoran]

Yes you are right. I changed the files:

# main.py
import argparse
import mymodule
import logging

logger = logging.getLogger(__name__)

def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Script for executing data 
quality checks.")
parser.add_argument("--log-level", default='info', choices=['notset', 
'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
args = parser.parse_args()

log_level = logging.getLevelName(args.log_level.upper())
logging.basicConfig(level=log_level, format=f"%(asctime)s - [%(levelname)s] 
- %(name)s - (%(filename)s).%(funcName)s(%(lineno)d) - %(message)s")
logger.setLevel(log_level)

log_some_messages()
mymodule.log_some_messages()

#mymodule.py
import logging

logger = logging.getLogger(__name__)


def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

and everything works as it should.
I was milsleaded by this article 
https://towardsdatascience.com/8-advanced-python-logging-features-that-you-shouldnt-miss-a68a5ef1b62d

Thanks.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43051] Pip Permissions Error overwriting pip directory

[Charley Riley]

New submission from Charley Riley :

After installing Python 3.9.1 on Windows 10 system-wide to "C:\Program 
Files\Python39", I tried to pip install the package 'dirsearch'.

pip refused with the error: "Defaulting to user installation because normal 
site-packages is not writeable", and proceeded to write to Local AppData 
(unwanted, so I deleted from AppData).

I changed the permissions of the site-packages folder to allow python access 
(allowed Users and Application Packages full control). 

Then I retried installing 'dirsearch', which now completed successfully. 
However, I was warned that my pip version was out of date. So I did:
>>> py -m pip install --upgrade pip
I ensured py --version = 3.9.1

This failed with "ERROR: Could not install packages due to an EnvironmentError: 
[WinError 5] Access is denied: 'c:\\program files\\python39\\scripts\\pip.exe'

I then allowed same permissions as site-packages to the scripts folder. 

Retried 
>>> py -m pip install --upgrade pip
and I get an error:
"C:\Program Files\Python39\python.exe: No module named pip"

I navigate to site-packages, where pip directories have been renamed as:
~ip
~ip-21.0.dist-info

--
components: Installation, Windows
messages: 385838
nosy: mcriley821, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Pip Permissions Error overwriting pip directory
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Glenn Linderman]

Glenn Linderman  added the comment:

Raymond said:
ISTM the scope of argparse was never intended to capture all possible patterns 
for command line argument parsing.  Instead, it aimed at to address the common 
cases.

I say:
Sounds like another wet blanket argpment. Refer to the section "Why aren't 
getopt and optparse enough?" in the argparse PEP 389.  It is clearn that 
argparse was intended to implement more complex cases than could be easily 
implemented with the prior getopt and optparse. The concepts of variable 
numbers of arguments for an option, and of subcommonds, introduced initially in 
argparse, are far more complex than the enhancements proposed here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Glenn Linderman]

Glenn Linderman  added the comment:

So the missing signature is why I didn't understand, probably. At least, it 
seems reasonable to blame that :)  You didn't include [version] in the 
signature, but that isn't your fault: it isn't in the original and should be 
(see action "version").

So key is optional and defaults to dest... I saw that, but was already 
confused.  This latest example clears up why you might want to overrride dest 
for use in different arguments... very similar to why dest is allowed to be 
specified instead of its default value.

And dest could be used, it is sufficient, but allowing specification of key to 
override it is more flexible, and could save user code in some circumstances, 
with little cost to the implementation.

Sounds good to me now.

I was also surprised by introduction of a "capture" action by itself, and look 
forward to documentation for it, as things progress. I can guess about 
store_capture and append_capture from the definition of extend_capture.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> capture mutual relationship between command line arguments for more
> advanced parsers, where order of command line arguments is meaningful

I don't think this is a pattern we should encourage.  It doesn't seem to arise 
often and it makes the API much more complex for end users.

Also, I'm concerned that adding more action classes will make argparse harder 
to learn for the average user.  Already, the module suffers from sprawl and has 
unexpected interactions between the components.  The docs and tutorial are 
voluminous and hard to digest in a single sitting.

ISTM the scope of argparse was never intended to capture all possible patterns 
for command line argument parsing.  Instead, it aimed at to address the common 
cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: For example: Question, moving a folder (T061RR7N1) containing a Specific file (ReadCMI), to folder: C:\\...\DUT0

[Jason Friedman]

>
>
> for path, dir, files in os.walk(myDestinationFolder):
> # for path, dir, files in os.walk(destfolder):
> print('The path is %s: ', path)
> print(files)
> os.chdir(mySourceFolder)
> if not os.path.isfile(myDestinationFolder + file):
> #  if not os.path.isfile(destfolder + file):
> print('The file is %s: ', file)
> shutil.copytree(mySourceFolder, myDestinationFolder)
> #  os.rename(path + '\\' + file, myDestinationFolder + file)
> #  os.rename(path + '\\' + file, destfolder + file)
> os.rename(path + '\\' + file, myDestinationFolder + file)
> elif os.path.isfile(myDestinationFolder + file):
> #  os.rename(path + '\\' + file, destfolder + file)
> shutil.copytree(mySourceFolder, myDestinationFolder)
> So I would very much appreciate your ideas on the above
> statements!Because...I think I've got the wrong function (os.path.isfile),
> when I should be (s/b) using a stepped approach!Note: program allows input
> of ID = T061RR7N1 (for example)1) find the folder containing "file": where
> folder = T061RR7N1, and file is "ReadCMI"; if TRUE, shutil.copytree
> C:\\...\T061RR7N1\ReadCMI (TO) C:\\...\DUT[?], where [?] is a num from 0 -
> 15.2) append to C:\\...\DUT[?]\T061RR7N1, which contains "ReadCMI"!
>
> and would you mind telling me why this works (in every example I've found
> on the internet): r'C:\\anyfolder\\anyotherfolder\\'...what does the "r"
> signify? If it's 'read', why can't I use the 'a' for append?
>

A few answers and then a question ...

"r" in this context is a raw literal, see for example
https://www.journaldev.com/23598/python-raw-string.

And you probably don't need it. Python and Windows are happy with
"c:/path/to/file.txt" for a file path.
Some of your code might look end up looking like:
full_path = parent + "/" + file

See also the os.path.join method (
https://docs.python.org/3/library/os.path.html#os.path.join) and the
pathlib module (
https://docs.python.org/3/library/pathlib.html#module-pathlib).

If you truly want to move the folder, as opposed to copying it, you can use
shutil.move.

Does the target folder already exist? What is the rule for determining what
the target folder should be named?
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak

[Martin Panter]

Change by Martin Panter :


--
keywords: +3.7regression

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43050] threading timer memory leak

[fengjiang]

fengjiang  added the comment:

yes, I find similar issues,the use the 
patch(https://github.com/python/cpython/pull/15228/files) to fix the bug

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43050] threading timer memory leak

[Martin Panter]

Martin Panter  added the comment:

Perhaps this is caused by Issue 37788. Python 3.7.4 introduced a leak for any 
thread that doesn't get its "join" method called. Timer is a subclass of 
Thread, so to confirm, see if calling "timer.join()" after "cancel" will make 
the leak go away.

--
nosy: +martin.panter
resolution:  -> duplicate
superseder:  -> fix for bpo-36402 (threading._shutdown() race condition) causes 
reference leak
type: performance -> resource usage

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion

[Zachary Ware]

Change by Zachary Ware :


--
nosy: +eric.araujo, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Tadek Kijkowski]

Tadek Kijkowski  added the comment:

> I'm a little confused by the mention of the "key" keyword argument. I suspect 
> that is an internal concept to argparse, possibly passed that way to internal 
> methods, but on the add_argument interface, it doesn't exist... instead there 
> is "name or flags" positional arguments, from which, together with the "dest" 
> argument, the "key" keyword argument is derived.  This is described under the 
> explanation for the "dest" parameter.

Hmm... that may be confusing. The "key" keyword argument is not internal 
concept, is new parameter that can be passed to add_argument, specifically for 
'capture' actions. The "key" name may be unfortunate, maybe it could be 
"dest_key" or "dict_key"? It can't be "dest", because it is goes to 
add_argument together with "dest". In most cases it is not needed and "dest" is 
used as dictionary key, but user may want to override that.
Like in this example:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("--user", default=None)
>>> parser.add_argument("--server", default="localhost")
>>> parser.add_argument("src1", action="store_capture", key="file", 
capture_reset=["user", "server"])
>>> parser.add_argument("src2", action="store_capture", key="file", 
capture_reset=["user", "server"])
>>> parser.add_argument("dst", action="store_capture", key="file", 
capture_reset=["user", "server"])
>>> parser.parse_args("first --user guest --server no_such second --server 
not_found third".split())
Namespace(user=None, server='localhost',
src1={'user': None, 'server': 'localhost', 'file': 'first'},
src2={'user': 'guest', 'server': 'no_such', 'file': 'second'},
dst={'user': None, 'server': 'not_found', 'file': 'third'})

The 'dest' is 'src1', 'src2' and 'dst' respectively, but we want to have 
unified layout of the dictionaries,
so 'key' is 'file' in all three cases.

Oh, and I forgot to update add_argument signature in rst earlier.

ArgumentParser.add_argument(name or flags..., [action], [nargs], \
   [const], [default], [type], [choices], [required], \
   [help], [metavar], [dest], [capture], \
   [capture_reset], [key])
...
...
   * dest - The name of the attribute to be added to the object returned by
 parse_args.

-> these are new:

   * capture - A name or a list of names of attributes to capture by one of
 capture actions.

   * capture_reset - A name or a list of name of attributes to capture and
 reset to default value by one of capture actions.

   * key - The key to use putting command-line argument to dictionary object
 created by capture actions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43050] threading timer memory leak

[fengjiang]

New submission from fengjiang :

Hi,we are transfering code from python2.7 to 3.7 and find that using 
threading.timer will cause memory leak. It works fine in python2.7 but not 3.7. 
To repreduce the problem, you can simply run the code below.

While True:
timer = threading.Timer(5, None)
timer.start()
timer.cancel()

you will find the memory of progress increases rapidly

--
components: Library (Lib)
messages: 385831
nosy: fengjiang
priority: normal
severity: normal
status: open
title: threading timer memory leak
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion

[Peter Donis]

Change by Peter Donis :


--
keywords: +patch
pull_requests: +23183
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24359

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1812] doctest _load_testfile function -- newline handling seems incorrect

[Peter Donis]

Change by Peter Donis :


--
pull_requests: +23184
pull_request: https://github.com/python/cpython/pull/24359

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43049] Use io.IncrementalNewlineDecoder for doctest newline conversion

[Peter Donis]

New submission from Peter Donis :

This is a follow-up to issue 1812:

https://bugs.python.org/issue1812

It was suggested in the discussion on that issue that the newline conversion in 
doctest that was corrected in that issue could be done using already defined 
resources in the io module instead of by a custom function. This issue is to 
provide an issue number for my pull request on github to do that: the io module 
provides the IncrementalNewlineDecoder object that does exactly what is needed.

--
components: Tests
messages: 385830
nosy: pdonis
priority: normal
severity: normal
status: open
title: Use io.IncrementalNewlineDecoder for doctest newline conversion
type: enhancement
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42973] argparse: mixing optional and positional arguments... not again

[Tadek Kijkowski]

Tadek Kijkowski  added the comment:

> So in the big picture, the purpose of this change is to treat the inputs like 
> a kind of state-machine.

Not necessarily that. Simple parsers should be easy to write, complicated 
parsers should be _possible_ to write. Users should be able to do whatever they 
wish by providing custom actions. Like writing Conway's Life in Excel or 
building Turing Machine using bricks and conveyors. Problem with positional 
parameter is a blocker. Fact that you can: 1. mix positional parameters with 
options in general, 2. have unlimited number of positional parameters, but 3. 
can't have 1 and 2 at the same time, makes great number of things impossible. 
For me that looks like a bug, not a feature.

Capture actions are second priority for me, because if this PR goes through, 
user will be able to write his own actions.

> While the proposed change to the core parser is (apparently) minor, it does 
> occur at the center of the action. That is not the place we want any (new) 
> bugs or backward incompatibility.

The core change is apparently minor, and also actually minor. It boils out to 
this:

@@ -2010,17 +2020,23 @@ def consume_positionals(start_index):
 match_partial = self._match_arguments_partial
 selected_pattern = arg_strings_pattern[start_index:]
 arg_counts = match_partial(positionals, selected_pattern)
+action_index = 0

 # slice off the appropriate arg strings for each Positional
 # and add the Positional and its args to the list
-for action, arg_count in zip(positionals, arg_counts):
+for arg_count in arg_counts:
+action = positionals[action_index]
 args = arg_strings[start_index: start_index + arg_count]
 start_index += arg_count
 take_action(action, args)
+# if positional action nargs is '**',
+# never remove it from actions list
+if action.nargs != AS_MANY_AS_POSSIBLE:
+action_index += 1

 # slice off the Positionals that we just parsed and return the
 # index at which the Positionals' string args stopped
-positionals[:] = positionals[len(arg_counts):]
+positionals[:] = positionals[action_index:]
 return start_index

It's not hard to determine that if all action.nargs != AS_MANY_AS_POSSIBLE, old 
and new code do exactly the same.
Besides that's what tests and code review is for. As for backward 
compatibility, nargs='**' was illegal until now,
and for all parameters with nargs!='**' the code behaves exactly as before.

> And the full implementation requires a new Action subclass that is quite 
> different from existing ones.

No, that's an extra. That's why it's separate issue and separate PR. This issue 
is about giving users ability to write their own actions.

> Given how different this is from the normal argparse parsing (and the POSIX 
> parsing argparse seeks to emulate), I question the wisdom of adding this, in 
> part or whole, to the stock distribution.  It could certainly be published as 
> a pypi.  That already has a number of  parsers, some built on argparse, 
> others stand alone.
>
> I also wonder whether it would be simpler to do this kind of parsing directly 
> from sys.argv.  Just step through that list, consuming the values and flags 
> in sequence.

In other words, "if argparse doesn't meet your requirements, you can write your 
own parser". I don't think that's the way to go.

Both GNU getopt and GNU argp_parse allow mixing options with positional 
arguments without reordering. For getopt it's: options argument string begins 
with a hyphen, for argp_parse it's ARGP_IN_ORDER flag. I don't see why argparse 
wouldn't allow that. Original POSIX getopt didn't even have longopts.

> Sorry to sound like a wet blanket, but I've seen too many seemingly innocent 
> patches that caused unforeseen problems down the line.

Yeah, that can be said about any project at any time - changes can introduce 
bugs, so let's just keep it as it is. I appreciate your ideas, but your 
criticism does sound unsubstantiated: "POSIX getopts didn't have it", "it can 
be done on pypi", "enhancements introduce bugs".

I think this change should be applied, because it is small change in code which 
removes significant obstacle in writing advanced option parsers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43048] Printing RecursionError results in RecursionError

[Vladimir Feinberg]

New submission from Vladimir Feinberg :

Python's standard logger provides an exception() method, which is to be called 
from except blocks to log out exception traces by pulling from sys.exc_info().

Per https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L617 , 
logger.exception('my message') eventually boils down to something like 
traceback.print_exception(ei[0], ei[1], ei[2], None, some_buffer) where ei is 
the sys.exc_info().

However, the traceback code generates that printout by constructing a 
TracebackException recursively for every `except` context.

In turn, if a RecursionError was generated from an exception thrown across many 
except blocks, the TracebackException construction itself will have a 
RecursionError. This is particularly bad in cases where you'd want to capture 
this failure information, such as when you're printing the exception, since 
you'll never find out about the originating error.

Certain (well-used) libraries rely on try/except for control flow, and 
occasionally do recurse in their except blocks, so such exceptions are not 
hypothetical.

A solution to this might be to avoid constructing a TracebackException, and 
instead unwind traceback context using a while loop.

--
components: Library (Lib)
files: exc.py
messages: 385828
nosy: vlad2
priority: normal
severity: normal
status: open
title: Printing RecursionError results in RecursionError
versions: Python 3.6, Python 3.7, Python 3.9
Added file: https://bugs.python.org/file49769/exc.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Alan Gauld via Python-list]

On 27/01/2021 23:04, 2qdxy4rzwzuui...@potatochowder.com wrote:

> systems are more painful than others, but yes, some debugging
> environments are more painful than others, too.

Very true! but a good debugger is a godsend. Howevder...

> A well placed call to print (they're not "print statements" anymore!)
> can be much more enlightening and much faster than single stepping
> through code in a debugger,

If you are single stepping more than 2 or 3 lines then you
aren't using the debugger properly!

Use breakpoints(selected by brain power!) and watches. The watches
are like print statements but don;t modify the source code
(nothing to remove later!)

Anytime I find myself hitting the step button more that a few
times I know I'm not using the debugger effectively and rethink.

>  and seeing the output from the same print
> statement inside a loop can be much better than manually examining
> variables iteration after iteration and trying to remember what the
> value was before.

So don't do that, use the debugger. Most of them have conditional
breakpoints that will stop only when you need to.

> The best debugging tool, however, remains your brain.  

Amen to that, regardless of any software tool used.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Alan Gauld via Python-list]

On 27/01/2021 18:42, C W wrote:

> I'd like to know how the experts on here are approaching and debugging
> this.
> 
> Bonus if no debugger or breakpoint. Just the good ol' run the function and
> evaluate/print output for problems.

One option you may like and nobody seems to have mentioned
yet is to use the interactive prompt.

Simply run the program with the -i flag and the program will stay in the
interpreter at the >>> prompt. Similar to what you seem to be used to
doing in R...

>From there you can print the current value of variables, inspect objects
etc. I find it useful sometimes when I'm not on my desktop PC, although
usually I jump into winpdb or similar if I need that kind of support.

dubuggers and breakpoints combined with watchpoints are very powerful
tools and easy to learn. (Although I may be pre-disposed since my first
job after uni' was to white-box test 500K lines of C using a VAX command
line debugger in batch mode. I wrote over 100K lines of debugger commands!)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[Alan Gauld via Python-list]

On 27/01/2021 19:27, Dietmar Schwertberger wrote:

> Python is an interactive language. You can develop a lot while working 
> on a Python console. Then copy and paste into a program.

Absolutely, the humble interactive prompt is often overlooked
as a development tool. It's not as good as the "evaluate
expression" tool in the Smalltalk workspace, say, but it's close.

I habitually use 3 windows: - an editor, a shell command
line (to execute the code) and an interactive Python prompt.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Michael Torrie]

On 1/27/21 11:42 AM, C W wrote:
> For learning purposes, here's the files:
> https://www.dropbox.com/sh/a3iy40rcvib4uvj/AAADmlM2i6NquWC1SV0nZfnDa?dl=0
> 
> Yes, you are correct about "employee" and "person" discrepancies. For now,
> the list comprehension is where I get stuck.
> 
> I'd like to know how the experts on here are approaching and debugging
> this.
> 
> Bonus if no debugger or breakpoint. Just the good ol' run the function and
> evaluate/print output for problems.

Like I said, the key is in the traceback.  It told you exactly what the
problem was. And seeing your full code I can now tell you why.  There is
no "created_at" field in the person dict (I noticed it's now called neo
in the code you just posted).  The dict is created directly from JSON
and there is no "created_at field anywhere in the JSON, which is why
you're getting that error.

I don't really understand much of the code you posted. There doesn't
seem to be any link between the MySQL database and the NEODatabase class
and instances.  Something is missing and it looks vaguely java-esque,
which may not be your best way to work in Python. I'm not sure what
you're trying to do so I can't really say much about it.

If you want to play with a database abstraction, there are several
libraries available, including SQLchemy.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42973] argparse: mixing optional and positional arguments... not again

[Glenn Linderman]

Glenn Linderman  added the comment:

paul j3 said:

Given how different this is from the normal argparse parsing (and the POSIX 
parsing argparse seeks to emulate), I question the wisdom of adding this, in 
part or whole, to the stock distribution.  It could certainly be published as a 
pypi.  That already has a number of  parsers, some built on argparse, others 
stand alone.


I say:

This has been a deficiency of argparse from day one. Tadek's solution seems to 
enable addressing the deficiency in a backward-compatible manner. Do you, paul, 
find any test failures? Or any incompatibilities that may not be in the test 
cases? If not, then it certainly does seem like a wet-blanket comment.


paul j3 forther said:

I also wonder whether it would be simpler to do this kind of parsing directly 
from sys.argv.  Just step through that list, consuming the values and flags in 
sequence.  


I say:

The whole point of argparse, and of deprecation of the prior optparse was to 
make more functionality available in a more powerful API. Increasing the power 
of the API seems to be consistent with the purpose of argparse. It took me some 
time and use cases to discover the limitations of argparse, and although 
parse_intermixed_args solved the use cases I had, I was well aware that it 
didn't solve cases of every existing Unix utility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Glenn Linderman]

Glenn Linderman  added the comment:

I'm a little confused by the mention of the "key" keyword argument. I suspect 
that is an internal concept to argparse, possibly passed that way to internal 
methods, but on the add_argument interface, it doesn't exist... instead there 
is "name or flags" positional arguments, from which, together with the "dest" 
argument, the "key" keyword argument is derived.  This is described under the 
explanation for the "dest" parameter.

But in the user documentation, the use of "key" is a surprise (to me, maybe I 
missed something).

Perhaps using "dest" rather than "key" in this documentation would be better? 
Or perhaps there is a precedent in the argparse documentation for what to call 
it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43047] logging.config formatters documentation is out of sync with code

[Ian Wienand]

Change by Ian Wienand :


--
keywords: +patch
pull_requests: +23182
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24358

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43047] logging.config formatters documentation is out of sync with code

[Ian Wienand]

New submission from Ian Wienand :

The dict based configuration does not mention the "class" option, and neither 
the ini-file or dict sections mention the style tag.

--
components: Library (Lib)
messages: 385825
nosy: iwienand
priority: normal
severity: normal
status: open
title: logging.config formatters documentation is out of sync with code
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Tadek Kijkowski]

Change by Tadek Kijkowski :


--
keywords: +patch
pull_requests: +23181
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24357

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[J. Pic]

Also

- https://github.com/cool-RR/pysnooper
- https://github.com/andy-landy/traceback_with_variables
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42979] _zoneinfo: zoneinfomodule_exec() doesn't check for PyDateTime_IMPORT failure

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23180
pull_request: https://github.com/python/cpython/pull/24356

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Tadek Kijkowski]

Change by Tadek Kijkowski :


--
nosy: +paul.j3, r.david.murray, rhettinger, v+python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43046] argparse: capturing actions

[Tadek Kijkowski]

New submission from Tadek Kijkowski :

This is spinoff from issue42973.

The purpose of this issue is to provide set of additional action classes for 
argparse, that could capture mutual relationship between command line arguments 
for more advanced parsers, where order of command line arguments is meaningful. 
It is hoped that this will work together with enhancement introduced in 
issue42973, providing also ability to capture relationship between options and 
positional parameters.

There will be four new action classes: 'extend_capture', 'append_capture', 
'store_capture' and 'capture'.

Here are excerpts from documentation being prepared for the new action classes 
and an example of use:

* 'extend_capture' - Captures preceding argument values listed in the
  'capture' and 'capture_reset' keyword arguments and creates a dictionary 
object
  from those values, then adds actual command-line value to the dictionary using
  the 'key' keyword argument, finally the created dictionary is appended to the
  'dest' list. After capturing, all arguments listed in 'capture_reset' are 
reset
  to their default values. If there are more than one command-line values, a new
  dictionary object is created, and appended to the list for each value, with
  'capture_reset' arguments reset to their default values after first value is
  added.
  Example usage:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("--load-addr", type=lambda s: int(s, 16))
>>> parser.add_argument("--exec-addr", type=lambda s: int(s, 16))
>>> parser.add_argument("--replace", action="store_true")
>>> parser.add_argument("--file", nargs="*", action="extend_capture",
... capture="*", capture_reset=["load_addr", "exec_addr"])
>>> parser.parse_args("--replace --load-addr 1900 --exec-addr 8023 "
... "--file CALC !BOOT".split())
Namespace(load_addr=None, exec_addr=None, replace=True, file=[
{'replace': True, 'exec_addr': 32803, 'load_addr': 6400, 'file': 
'CALC'},
{'replace': True, 'exec_addr': None, 'load_addr': None, 'file': 
'!BOOT'}])

capture

The list of attributes to capture. This can be either a single attribute name,
a list (or other iterable type) of names or special value '*'. Name of
attribute associated with each argument is determined by the dest keyword
passed to add_argument method when the argument was
created. If capture_ is '*', all attributes are captured, except for this
argument's own value.

This keyword argument is valid only for 'extend_capture',
'append_capture', 'store_capture' and 'capture' actions.


capture_reset

The list of attributes to capture and reset to default value. As with capture,
this can be '*' to capture and reset all attributes except for this
argument's own value.

This keyword argument is valid only for 'extend_capture',
'append_capture', 'store_capture' and 'capture' actions.


key

The key to use for adding this argument's own command-line value to dictionary
of captured values. If this keyword argument is not specified, the dest is
used.

This keyword argument is valid only for 'extend_capture',
'append_capture' and 'store_capture' actions.

--
components: Library (Lib)
messages: 385824
nosy: monkeyman79
priority: normal
severity: normal
status: open
title: argparse: capturing actions
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[duncan smith]

On 27/01/2021 22:41, C W wrote:
> Great tutorial Irv, very simple with if-else example, gets the point
> across.
> 
> My main takeaway from the discussion so far is that: you can't troubleshoot
> Python without some kind of breakpoint or debugger.
> 

[snip]

Really?

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43045] Instructions for installing pip on Ubuntu/WSL2 is incomplete

[Lennart Borgman]

New submission from Lennart Borgman :

Please see here for details:

https://stackoverflow.com/questions/65928875/no-module-named-pip-python-3-9/65929009#65929009

--
messages: 385823
nosy: lennart.borgman
priority: normal
severity: normal
status: open
title: Instructions for installing pip on Ubuntu/WSL2 is incomplete
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43044] Python 2.7 documentation links to 404 pages when the library was moved or renamed

[STINNER Victor]

STINNER Victor  added the comment:

The Python 2.7 branch has been removed: 
https://github.com/python/cpython/pull/24195#issuecomment-768474389

I close the issue. Please use the Python 3 documentation.

--
nosy: +vstinner
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43044] Python 2.7 documentation links to 404 pages when the library was moved or renamed

[Борис Верховский]

New submission from Борис Верховский :

1. go to https://docs.python.org/2/library/stringio.html (note the "/2/")
2. click on "You should upgrade and read the Python documentation for the 
current stable release." at the top

I expect to be sent to the Python 3 documentation for StringIO (or at least the 
index.html page of the Python 3 documentaion) but because StringIO was moved 
from its own library into `io`, the Python 2 docs are linking to a 404 error 
page, because they just rewrite "/2/" to "/3/" in the URL.

See attached PR for a list of all these broken pages and where they could 
redirect to.

--
assignee: docs@python
components: Documentation
messages: 385821
nosy: boris, docs@python
priority: normal
pull_requests: 23179
severity: normal
status: open
title: Python 2.7 documentation links to 404 pages when the library was moved 
or renamed
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43040] random.py randrange() is very slow if the range is a power of 2.

[Donovan Baarda]

Donovan Baarda  added the comment:

I had a look at the C implementation of getrandbits() and spotted this;

https://github.com/python/cpython/blob/64fc105b2d2faaeadd1026d2417b83915af6622f/Modules/_randommodule.c#L487

In my case I was also being hit by randrange(2**32) calling getrandbits(33) 
just pushing it past using the optimized fast-path for ranges needing <=32 
bits. So not only was it calling getrandbits() 2x as many times as necessary, 
under the hood it was generating 64 random bits each time using a slow path.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38307] Add .end_lineno attribute to pyclbr _Objects

[Terry J. Reedy]

Terry J. Reedy  added the comment:

If IDLE were to recalculate the end from start, it would have to more or less 
parse forward to find the end of the statement.  This is why I considered it a 
speculative wish.

> would `end_lineno=None` make any sense?  
Only as a kludge if a default were required because of the signature position.

Point 3: The existing doc implies but does not exactly state that the _Object 
signature are private, so that we could break any possibly existing (but 
possibly not, and discouraged) use.  I think back-compatibility is discussed in 
the devguide.

Posts do not have a link until distributed, and then you can find it on 
mail.python.org.  The title is sufficient to do that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I'm pretty sure this is fixed for 3.8+.

whether or not it should be considered a bugfix and backported to 3.7.x is 
probably too late at this point in release lifecycles anyways.

thanks for raising this and the fixing PR!

--
nosy: +gregory.p.smith
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[2QdxY4RzWzUUiLuE]

On 2021-01-27 at 17:41:52 -0500,
C W  wrote:

> Great tutorial Irv, very simple with if-else example, gets the point
> across.

Disclaimer:  I did not watch the video.

> My main takeaway from the discussion so far is that: you can't
> troubleshoot Python without some kind of breakpoint or debugger.

I disagree.  :-)

Having spent a long time developing and debugging embedded systems (some
soft- or hard- real time), I believe that you can troubleshoot programs
(regardless of source language) without debuggers.  If a system can
generate output, then I (the programmer) can change that output, observe
the results, and figure out what's working and what's not.  Yes, some
systems are more painful than others, but yes, some debugging
environments are more painful than others, too.

A well placed call to print (they're not "print statements" anymore!)
can be much more enlightening and much faster than single stepping
through code in a debugger, and seeing the output from the same print
statement inside a loop can be much better than manually examining
variables iteration after iteration and trying to remember what the
value was before.

The best debugging tool, however, remains your brain.  Way before I add
a call to print or you set up your debugger, thinking about what went
wrong and where to look can solve the problems without resorting to
external tools.  :-)
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42955] Add sys.stdlib_module_names: list of stdlib module names (Python and extension modules)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 64fc105b2d2faaeadd1026d2417b83915af6622f by Victor Stinner in 
branch 'master':
bpo-42955: Remove sub-packages from sys.stdlib_module_names (GH-24353)
https://github.com/python/cpython/commit/64fc105b2d2faaeadd1026d2417b83915af6622f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38307] Add .end_lineno attribute to pyclbr _Objects

[Aviral Srivastava]

Aviral Srivastava  added the comment:

Hey Terry, thanks for commenting. I have a few questions to ask you, please 
pardon my lack of awareness.

>This requieres end lines and having it be an attribute is easier and more 
>accurate than recalculating it.

How do you recalculate the end_lineno?


Since all the objects that _start_, have an _end_ too, would `end_lineno=None` 
make any sense?


I could not make anything out of your third point where you mention `readline` 
and `readline_ex`. Can you explain that point?


>I posted "What is the pyclbr public API" to pydev asking about this issue.

Can you share the link of your post?


As for my use case, I need the scope of a class and a function in the new tool 
that I am developing. My tool is to generate a new type of UML diagrams for a 
given codebase. Do check it out and leave your critical feedback: 
https://github.com/kebab-mai-haddi/gruml

I need to know about the scope (start and end) so that I can deduce whether the 
"used_at" attribute of an object is within a class. This will help me deduce 
whether a particular class (or a function) uses any imported object. This is to 
deduce the dependency of a class on other classes and functions.

Currently, I have to edit the pyclbr and make it custom, then, had to Dockerize 
the whole thing. So, want to contribute to `cpython` so that this feature is 
present at the source itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[J. Pic]

You're most welcome flaskee, however I would recommend you try them all.
Thonny in particular because it distinguishes line from expression which I
believe is a unique feature.

Personally I have learned to love just pdb, and a whiteboard!

Le mer. 27 janv. 2021 à 23:03, flaskee  a écrit :

>
> Thank you J. Pic.
>
> Out of everything today,
> (and given my priority is Python/Flask debugging)
> it looks like Wing IDE is something to dig into.
>
> Thanks
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐ Original Message ‐‐‐
> On Wednesday, January 27, 2021 4:09 PM, J. Pic  wrote:
>
> > Thonny, winpdb/winpdb-rebord, eric4, pudb, web-pdb, vy, mu, netbeans,
> > eclipse, pdbpp...
> >
> > Also see: https://wiki.python.org/moin/PythonDebuggingTools
> >
> > "Changing a variable" -> that's basically evaluating code ? -> supported
> in
> >
> > all debuggers I suppose
> >
> > 
> >
> > https://mail.python.org/mailman/listinfo/python-list
>
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43040] random.py randrange() is very slow if the range is a power of 2.

[Donovan Baarda]

Donovan Baarda  added the comment:

FTR, more articles about the slowness of generating random integers in Python;

https://lemire.me/blog/2016/03/21/ranged-random-number-generation-is-slow-in-python/

https://www.reddit.com/r/Python/comments/jn0bb/randomrandint_vs_randomrandom_why_is_one_15x/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[C W]

Great tutorial Irv, very simple with if-else example, gets the point
across.

My main takeaway from the discussion so far is that: you can't troubleshoot
Python without some kind of breakpoint or debugger.

I suppose I can't take the functional programming debugger style like C,
Matlab, or R, and apply it to a OOP language like Python.

On Wed, Jan 27, 2021 at 5:26 PM Irv Kalb  wrote:

> On Jan 26, 2021, at 5:28 PM, William Ray Wing via Python-list <
> python-list@python.org> wrote:
> >
> >
> >
> >> On Jan 26, 2021, at 2:00 PM, C W  wrote:
> >>
> >> Hello everyone,
> >>
> >> I'm a long time Matlab and R user working on data science. How do you
> >> troubleshooting/debugging in Python?
> >>
> >
> > Another approach is to run the code in an IDE.  I happen to use Wing,
> but that is a coincidence.  But almost ANY IDE will let you set a break
> point, then single-step through your code starting at the break point and
> examine the values of your variables at each step.  Sometimes this is an
> awfully big hammer for what is a head-slapping mistake.  But it has never
> failed me.
> >
> >
>
> I'm happy with the PyCharm IDE.  I created a video showing how to use the
> debugger in that environment.  It's available on YouTube here:
>
> https://www.youtube.com/watch?v=cxAOSQQwDJ4  >
>
> Irv
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue38307] Add .end_lineno attribute to pyclbr _Objects

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The answer, Aviral discovered,  is that current ast nodes have end_lineno 
attributes, so the patch amounts to consistently copying them.

I don't understand Aviral's use case either, but an item on my IDLE wish list 
is to be able to move class and function definitions within a file by moving 
the Class and Function entries within the browser tree.  This requieres end 
lines and having it be an attribute is easier and more accurate than 
recalculating it.  Merely clicking on an entry could highlight the whole 
definition.  Other people might think of other uses.

An issue in the patch is inserting parameter 'end_line' after 'line'.  
Normally, new parameters have to go at the end, but:

1. Inserting 'end_line' after 'line' makes the code easier to read.

2. The parameter or parameters at the end have default values.  If we added 
'end_line=None', we would have to consider what to do when it is omitted.

3. https://docs.python.org/3/library/pyclbr.html#module-pyclbr, which I rewrote 
in 2017, documents readline and readline_ex as the public call interface.  They 
return a hierarchical structure that includes Functions and Classes in the dict 
values.  Their signatures are intentionally omitted and the attributes are only 
listed after saying "Users are not expected to create instances of these 
classes."

I posted "What is the pyclbr public API" to pydev asking about this issue.

--
nosy: +terry.reedy
title: Provide Class' end line in pyclbr module -> Add .end_lineno attribute to 
pyclbr _Objects
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

For example: Question, moving a folder (T061RR7N1) containing a Specific file (ReadCMI), to folder: C:\\...\DUT0

[Kevin M. Wilson via Python-list]

for path, dir, files in os.walk(myDestinationFolder):
# for path, dir, files in os.walk(destfolder):
print('The path is %s: ', path)
print(files)
os.chdir(mySourceFolder)
if not os.path.isfile(myDestinationFolder + file):
#  if not os.path.isfile(destfolder + file):
print('The file is %s: ', file)
shutil.copytree(mySourceFolder, myDestinationFolder)
#  os.rename(path + '\\' + file, myDestinationFolder + file)
#  os.rename(path + '\\' + file, destfolder + file)
os.rename(path + '\\' + file, myDestinationFolder + file)
elif os.path.isfile(myDestinationFolder + file):
#  os.rename(path + '\\' + file, destfolder + file)
shutil.copytree(mySourceFolder, myDestinationFolder)
So I would very much appreciate your ideas on the above statements!Because...I 
think I've got the wrong function (os.path.isfile), when I should be (s/b) 
using a stepped approach!Note: program allows input of ID = T061RR7N1 (for 
example)1) find the folder containing "file": where folder = T061RR7N1, and 
file is "ReadCMI"; if TRUE, shutil.copytree C:\\...\T061RR7N1\ReadCMI (TO) 
C:\\...\DUT[?], where [?] is a num from 0 - 15.2) append to 
C:\\...\DUT[?]\T061RR7N1, which contains "ReadCMI"!

and would you mind telling me why this works (in every example I've found on 
the internet): r'C:\\anyfolder\\anyotherfolder\\'...what does the "r" signify? 
If it's 'read', why can't I use the 'a' for append?
KMW

"The only way to have experience is by having the experience"!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Irv Kalb]

On Jan 26, 2021, at 5:28 PM, William Ray Wing via Python-list 
 wrote:
> 
> 
> 
>> On Jan 26, 2021, at 2:00 PM, C W  wrote:
>> 
>> Hello everyone,
>> 
>> I'm a long time Matlab and R user working on data science. How do you
>> troubleshooting/debugging in Python?
>> 
> 
> Another approach is to run the code in an IDE.  I happen to use Wing, but 
> that is a coincidence.  But almost ANY IDE will let you set a break point, 
> then single-step through your code starting at the break point and examine 
> the values of your variables at each step.  Sometimes this is an awfully big 
> hammer for what is a head-slapping mistake.  But it has never failed me.
> 
> 

I'm happy with the PyCharm IDE.  I created a video showing how to use the 
debugger in that environment.  It's available on YouTube here:

https://www.youtube.com/watch?v=cxAOSQQwDJ4 

Irv

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[flaskee via Python-list]

Thank you J. Pic.

Out of everything today,
(and given my priority is Python/Flask debugging)
it looks like Wing IDE is something to dig into.

Thanks


Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, January 27, 2021 4:09 PM, J. Pic  wrote:

> Thonny, winpdb/winpdb-rebord, eric4, pudb, web-pdb, vy, mu, netbeans,
> eclipse, pdbpp...
>
> Also see: https://wiki.python.org/moin/PythonDebuggingTools
>
> "Changing a variable" -> that's basically evaluating code ? -> supported in
>
> all debuggers I suppose
>
> 
>
> https://mail.python.org/mailman/listinfo/python-list


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43043] Python tutorial could make rules for default argument definition clearer

[Jesse Silverman]

New submission from Jesse Silverman :

I love the tutorial, but have been pointing out things I only knew from brutal 
experience, other tutorials, reasoning from other languages I know better, or 
looking things up.  I only mention them if they seem in scope and at the same 
level of the material which is presented.

Regarding 4.7.1. Default Argument Values of the excellent Python tutorial:

I guessed that I would see the behavior shown below from my understanding of 
default arguments in several other languages.  I did not read anything in this 
tutorial section that would lead me to conclude this, and other things 
mentioned in this section are neither much more or less subtle, more or less 
obvious, or more or less important.

PS F:\PythonCode> py VarArgs.py
  File "F:\PythonCode\VarArgs.py", line 1
def varArgs(quote, footnote='', times):
 ^
SyntaxError: non-default argument follows default argument

Most introductions to default argument definition in other languages go out of 
their way to highlight that the default parameters can be only the last N of 
them, and that all required ones must come first.
Python is intended to be accessible to people no more clever than the other 
languages intend, so tho it seems obvious to me that they must only come after 
all required arguments, I remember when it was not obvious (and fortunately the 
tutorials I read in the past for other languages belabored the point).

The explanatory text of:
giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2)

as it merely says "one of", rather than "the first of" led me to wonder if 
Python was somehow more flexible/better at mind-reading than the languages I 
already know well that require all default args to be trailing, and to only let 
you default the last one, two, or ... N.  The results of the experiment shown 
above demonstrate it is similar to the behavior in other languages in this 
regard.

I deeply appreciate the information about when the default arguments get 
evaluated, which is highly relevant and is not something I would have picked up 
on from other languages I've used.

A simple line stating that "any default arguments must follow any required 
arguments in the definition" and something like "on calls, optional arguments 
may only be omitted from the end of the parameter list, they can't simply be 
skipped" would be awesome.

I am not sure whether or not this is made perfectly clear in the next section, 
which has a lot of other new and interesting ideas flying around in it as well, 
besides this one.

--
assignee: docs@python
components: Documentation
messages: 385813
nosy: docs@python, jessevsilverman
priority: normal
severity: normal
status: open
title: Python tutorial could make rules for default argument definition clearer
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to accept argparse.log_level parameter and propagate its value to all other modules

[Peter Otten]

On 27/01/2021 20:04, Zoran wrote:

In the same folder I have three files: main.py, app_logger.py and mymodule.py 
with their contents:

# main.py
import argparse
import app_logger
import mymodule
import logging

logger = app_logger.get_logger(__name__)

def log_some_messages():
 logger.debug(f'{__name__} - debug message')
 logger.info(f'{__name__} - info message')
 logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
 parser = argparse.ArgumentParser(description="Script for executing data quality 
checks.")
 parser.add_argument("--log-level", default='info', choices=['notset', 'debug', 
'info', 'warning', 'error', 'critical'], help="set log level")
 args = parser.parse_args()

 logger.setLevel(eval(f"logging.{args.log_level.upper()}"))

 log_some_messages()
 mymodule.log_some_messages()


#mymodule.py
import logging

from script_logging import app_logger

logger = app_logger.get_logger(__name__)


def log_some_messages():
 logger.debug(f'{__name__} - debug message')
 logger.info(f'{__name__} - info message')
 logger.warning(f'{__name__} - warning message')


#app_logger.py
import logging

_log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - 
(%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
_log_level = logging.DEBUG

def get_stream_handler():
 stream_handler = logging.StreamHandler()
 stream_handler.setLevel(_log_level)
 stream_handler.setFormatter(logging.Formatter(_log_format))
 return stream_handler

def get_logger(name):
 logger = logging.getLogger(name)
 logger.setLevel(_log_level)
 logger.addHandler(get_stream_handler())
 return logger

When I run it with main.py --log-level debug I get the following output:

2021-01-25 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) 
- __main__ - debug message
2021-01-25 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - 
__main__ - info message
2021-01-25 13:21:01,152 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:21:01,152 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:21:01,152 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:21:01,152 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

which is OK. If I change --log-level parameter to 'warning', than output is the 
following:

2021-01-25 13:22:12,760 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:22:12,760 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:22:12,761 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:22:12,761 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

As you can see, in main.py log level is set correctly, but in mymodule.py it is 
not.

How to accept argparse.log_level parameter in if __name__ == '__main__': and 
propagate its value to all other modules?


You just have to set the level in the root logger. The easiest way to
achieve that is to forget about your app_logger wrapper around logging
and call

logging.basicConfig(level=yourlevel, format=yourformat)

in  main.py and to create the loggers in main and mymodule

with

logger = logging.getLogger(__name__)


--
https://mail.python.org/mailman/listinfo/python-list

Re: imalib cant get From: adresses

[Barry Scott]

Sigh you use a damaged email address for reply does not work.

> On 27 Jan 2021, at 14:30, Bischoop  wrote:
> 
> 
> I don't have much experience with imaplib and for a few days can't
> manage to get properly, I followed also some tutorial but also met few
> problems.
> What I want is to get email addresses from emails I do have in gmail
> inbox. The code below gets them but then pops an error.
> When I change the part "for i in range()" as in commented line it gets
> just the last email.
> 
> Anybody familiar here with imaplib? 

You have assumed that message ID are integers and in order.
Where as they are assigned by the IMAP server and are not in order.
I'm not even sure that they need to be integers.

You need to search the for the messages you want to get back a list of msg_id's.

The you can use the loop over the msg id doing operations on them.

After you M.select() call M.search() with appropriate params.
search is very powerful and will get you only the messages you are interested 
in.

Here is a fragment of some of my code to show the idea:

   folder_info = self.server.select_folder( folder, readonly=True )
   msg_ids = self.server.search( ['ALL'] )
   print( msg_ids )

   all_msg_info = self.server.fetch( msg_ids, ['FLAGS','ENVELOPE'] )
   for msg_id in all_msg_info:
   msg_info = all_msg_info[ msg_id ]
   print( '%-40s %s %s' % (msg_info[b'FLAGS'], 
msg_info[b'ENVELOPE'].date, msg_info[b'ENVELOPE'].subject) )

Barry

> 
> 
> import getpass, imaplib
> import email
> from email.header import decode_header
> 
> M = imaplib.IMAP4_SSL('imap.gmail.com')
> M.login(u, p)
> M.select()
> status, messages = M.select("INBOX")
> di={}
> 
> # number of top emails to fetch
> N = 100
> 
> # total number of emails
> messages = int(messages[0])
> 
> for i in range(messages, messages-N,-1):
> #for i in range(messages, N):
>   # fetch the email message by ID
>   res, msg = M.fetch(str(i), "(RFC822)")
>   for response in msg:
>   if isinstance(response, tuple):
>   # parse a bytes email into a message object
>   msg = email.message_from_bytes(response[1])
> 
> 
>   From, encoding = decode_header(msg.get("From"))[0]
>   if isinstance(From, bytes):
>   From = From.decode(encoding)
>   print("From:", From)
>   print(type(From))
> 
> 
>   print("="*100)
>   if '<'in From:
>   s = From.index('<')
>   d = From.index('>')
>   q = From[s + 1:d]
>   w = From[0:s - 1]
>   if q in di.values():
>   pass
>   else:
>   di[w] = q
>   else:
>   if q not in di.values():
> 
>   b = sum(1 for key in di if key.startswith('Friend'))
>   di[f'Friend{b+1}']=From
> # close the connection and logout
> M.close()
> M.logout()
> 
> print(di)
> for i in di.keys():
>   if 'Friend' in i:
>   print(From,'\n',i,di[i])
> 
> b = sum(1 for key in di if key.startswith('Friend'))
> print(b)
> 
>   raise self.error('%s command error: %s %s' % (name, typ, data))
> imaplib.error: FETCH command error: BAD [b'Could not parse command']
> 
> Process finished with exit code 1
> 
> 
> 
> 
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue41282] Deprecate and remove distutils

[Steve Dower]

Steve Dower  added the comment:

That PR is just to add the import warning and update docs. I want to make sure 
that's in asap so we don't miss the release.

Is there anywhere else in the docs that needs a note? Distutils has been marked 
as deprecated for years already, so it's really just emphasising that and 
adding the 3.12 removal date. I think whatsnew, library and the old doc 
sections is enough, yeah?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[J. Pic]

Thonny, winpdb/winpdb-rebord, eric4, pudb, web-pdb, vy, mu, netbeans,
eclipse, pdbpp...

Also see: https://wiki.python.org/moin/PythonDebuggingTools

"Changing a variable" -> that's basically evaluating code ? -> supported in
all debuggers I suppose
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue41282] Deprecate and remove distutils

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +23178
pull_request: https://github.com/python/cpython/pull/24355

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Problems that you are going to find:

* The c tokenizer throws syntax errors while the tokenizer module does not. For 
example:

❯ python -c "1_"
  File "", line 1
1_
 ^
SyntaxError: invalid decimal literal

❯ python -m tokenize <<< "1_"
1,0-1,1:NUMBER '1'
1,1-1,2:NAME   '_'
1,2-1,3:NEWLINE'\n'
2,0-2,0:ENDMARKER  ''

* The encoding cannot be immediately specified. You need to thread it in many 
places.

* The readline() function can now return whatever or be whatever, that needs to 
be handled (better) in the c tokenizer to not crash.

* str/bytes in the c tokenizer.

* The c tokenizer does not get the full line in some cases or is tricky to get 
the full line.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42973] argparse: mixing optional and positional arguments... not again

[paul j3]

paul j3  added the comment:

Sometimes patches have unforeseen benefits.  My earlier patch for this issue, 
parse_intermixed_args, has been useful beyond the OP's case.

https://stackoverflow.com/questions/50916124/allow-positional-command-line-arguments-with-nargs-to-be-seperated-by-a-flag

https://bugs.python.org/issue15112
argparse: nargs='*' positional argument doesn't accept any items if preceded by 
an option and another positional

With

 usage: test.py [-h] [-a A] b [c]

and 

 test.py B -a A C

has problems because the optional '[c]' positional is used up when 'b' is 
processed.   Intermixed gets around this by first processing '-a', and then 
handling 'b [c]' together.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42973] argparse: mixing optional and positional arguments... not again

[paul j3]

paul j3  added the comment:

So in the big picture, the purpose of this change is to treat the inputs like a 
kind of state-machine.

In the bigger example, the `**` positional values are processed one by one, 
using the interspersed optionals to set state parameters.

`args.sources` then ends up as a list of dicts, each of the form:

{'name': 'input3.txt', 'frobnicate': None, 'massage_level': 5}

where 'name' is the positional value, and 'frobnicate' and 'massage_level' are 
the latest values (default or user supplied).  The optionals can be specified 
in any order or repeatedly.

While the proposed change to the core parser is (apparently) minor, it does 
occur at the center of the action.  That is not the place we want any (new) 
bugs or backward incompatibility.  And the full implementation requires a new 
Action subclass that is quite different from existing ones.

Given how different this is from the normal argparse parsing (and the POSIX 
parsing argparse seeks to emulate), I question the wisdom of adding this, in 
part or whole, to the stock distribution.  It could certainly be published as a 
pypi.  That already has a number of  parsers, some built on argparse, others 
stand alone.

I also wonder whether it would be simpler to do this kind of parsing directly 
from sys.argv.  Just step through that list, consuming the values and flags in 
sequence.  

Sorry to sound like a wet blanket, but I've seen too many seemingly innocent 
patches that caused unforeseen problems down the line.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have built a draft of how the changes required to make what you describe, in 
case you want to finish them:

https://github.com/pablogsal/cpython/tree/tokenizer_mod

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[Paul Bryan via Python-list]

My experience with IntelliJ (related to PyCharm): it scans all source
files in the project, compiles them, graphs all dependencies, compiles
those (if necessary) or inspects their class bytecode, and so on to
build a full graph in memory to support showing errors in real time
(highlighting in source) and autocomplete when user is typing. In my
experience, it's heavyweight, and creates more reliance on the features
of an IDE than I feel is healthy (IDE becomes a crutch). I suspect
PyCharm is doing much of the same.

On Wed, 2021-01-27 at 15:36 -0500, C W wrote:
> I don't know exactly, but it shows as inspection on the bottom left
> corner.
> 
> I believe it's indexing in the background.
> 
> On Wed, Jan 27, 2021 at 3:25 PM Grant Edwards
> 
> wrote:
> 
> > On 2021-01-27, C W  wrote:
> > > I'm not expert in Python, but I sure tried many IDEs to kick off
> > > Python
> > > programming.
> > > 
> > > I started with PyCharm, but I had a problem with it constantly
> > > scanning
> > the
> > > background, even after I turned that feature off.
> > 
> > What is it scanning?
> > 
> > > My favorite (I'm using now) is VS Code with Python extension,
> > > it's very
> > > light. Recently also started background scanning, but that's
> > > generally
> > done
> > > in 30 seconds.
> > 
> > Same question, what is it scanning?
> > 
> > --
> > Grant
> > 
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> > 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[C W]

I meant bottom right corner, not left. opps!

On Wed, Jan 27, 2021 at 3:36 PM C W  wrote:

> I don't know exactly, but it shows as inspection on the bottom left corner.
>
> I believe it's indexing in the background.
>
> On Wed, Jan 27, 2021 at 3:25 PM Grant Edwards 
> wrote:
>
>> On 2021-01-27, C W  wrote:
>> > I'm not expert in Python, but I sure tried many IDEs to kick off Python
>> > programming.
>> >
>> > I started with PyCharm, but I had a problem with it constantly scanning
>> the
>> > background, even after I turned that feature off.
>>
>> What is it scanning?
>>
>> > My favorite (I'm using now) is VS Code with Python extension, it's very
>> > light. Recently also started background scanning, but that's generally
>> done
>> > in 30 seconds.
>>
>> Same question, what is it scanning?
>>
>> --
>> Grant
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[C W]

I don't know exactly, but it shows as inspection on the bottom left corner.

I believe it's indexing in the background.

On Wed, Jan 27, 2021 at 3:25 PM Grant Edwards 
wrote:

> On 2021-01-27, C W  wrote:
> > I'm not expert in Python, but I sure tried many IDEs to kick off Python
> > programming.
> >
> > I started with PyCharm, but I had a problem with it constantly scanning
> the
> > background, even after I turned that feature off.
>
> What is it scanning?
>
> > My favorite (I'm using now) is VS Code with Python extension, it's very
> > light. Recently also started background scanning, but that's generally
> done
> > in 30 seconds.
>
> Same question, what is it scanning?
>
> --
> Grant
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[Grant Edwards]

On 2021-01-27, C W  wrote:
> I'm not expert in Python, but I sure tried many IDEs to kick off Python
> programming.
>
> I started with PyCharm, but I had a problem with it constantly scanning the
> background, even after I turned that feature off.

What is it scanning?

> My favorite (I'm using now) is VS Code with Python extension, it's very
> light. Recently also started background scanning, but that's generally done
> in 30 seconds.

Same question, what is it scanning?

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: imalib cant get From: adresses

[Bischoop]

On 2021-01-27, Bischoop  wrote:


Solved using IMAP4.uid.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue38307] Provide Class' end line in pyclbr module

[Aviral Srivastava]

Aviral Srivastava  added the comment:

I have made the changes and tested, my builds were successful. Please review 
it? 

CC @brandtbucher, @steven.daprano

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[flaskee via Python-list]

" also started background scanning, but that's generally done in 30 seconds."

Do we know what PyCharm is background scanning for?
Do we know what VS Code is scanning for?

I've been leery of VS* things since 2013, when Microsoft (secretly) changed 
their VS compiler, so that every single .exe, .dll, etc that was compiled; 
included a tiny API call, that dialed back to Microsoft. Even a Hello-World.exe 
would get this telemetry API embedded. So, if you distributed executables -- MS 
knew right where they went to, when the user ran them.

There was "outrage" when Microsoft was finally caught; and they backed off of 
this in the next version (I do not know if they've snuck it in again or not). 
But it did not raise my confidence in tools from Microsoft all that much.

I love Python.
I just want to add a real debugger ;-)

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, January 27, 2021 2:07 PM, C W  wrote:

> I'm not expert in Python, but I sure tried many IDEs to kick off Python 
> programming.
>
> I started with PyCharm, but I had a problem with it constantly scanning the 
> background, even after I turned that feature off.
>
> My favorite (I'm using now) is VS Code with Python extension, it's very 
> light. Recently also started background scanning, but that's generally done 
> in 30 seconds.
>
> On Wed, Jan 27, 2021 at 1:51 PM Michał Jaworski  wrote:
>
>> PyCharm has all these debugging capabilities and there is a community 
>> edition that you can use for free. If you earn for the living with Python it 
>> is worth investing in professional edition though.
>>
>> Michał Jaworski
>>
>>> Wiadomość napisana przez flaskee via Python-list  w 
>>> dniu 27.01.2021, o godz. 19:32:
>>>
>>> 
>>> While print() is groovy and all,
>>> if anyone runs across a non-pdb python debugger (standalone or IDE-based)
>>> please let me know.
>>>
>>> I too was blessed with IDE-based debugging (in the 90's!)
>>> * where you can set break point(s);
>>> * have the program stop right before a suspected failure point;
>>> * check the contents of ALL variables, and choose whether to restart;
>>> * or skip a few lines before restarting;
>>> * or change a variable (hot, move back a few lines and restart, etc.
>>> * Some, would even let you alter the code a bit before restarting.
>>>
>>> I too, miss this.
>>>
>>>
>>> Hopefully I did not miss someone mentioning
>>> such a python tool in the prior thread.
>>>
>>> Thanks!
>>>
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDE tools to debug in Python?

[Dietmar Schwertberger]

On 27.01.2021 20:07, C W wrote:

I'm not expert in Python, but I sure tried many IDEs to kick off Python
programming.

I started with PyCharm, but I had a problem with it constantly scanning the
background, even after I turned that feature off.

My favorite (I'm using now) is VS Code with Python extension, it's very
light. Recently also started background scanning, but that's generally done
in 30 seconds.


There's also Wing IDE (Pro). The best is to try at least PyCharm and 
Wing IDE (Pro) and maybe one or more of the others.


Besides the debugger experience from the 90's please note that stepping 
through a program is not the most valuable feature of the Python debuggers:
Python is an interactive language. You can develop a lot while working 
on a Python console. Then copy and paste into a program.


Especially for hardware control, network programming and GUI development 
I find that the most valuable feature of Python.
(Maybe you have used Forth in the 90s as well, then you will know this 
feature.)


The better debuggers have a built-in console.
See here for a screenshot of Wing's debug console exploring GUI code: 
http://wxglade.sourceforge.net/docs/_images/Calculator_Debugger.png


Regards,

Dietmar



--
https://mail.python.org/mailman/listinfo/python-list

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[Dietmar Schwertberger]

On 27.01.2021 01:52, Skip Montanaro wrote:

Agree with Grant on these points. I certainly use gdb to debug C code
(like the interpreter), but for Python code, tracebacks and print
statements pretty much take care of things.

I thought so for the first 12 year of using Python.
For the last 12 years I have been using an IDE/debugger with built-in 
console.


It boosted productivity by at least a factor of two.


Sometimes "python -i ..." and "import pdb" is good enough, being able to 
inspect variables after an exception.

I have the impression that pdb as post mortem debugger is not well known...

Regards,

Dietmar


--
https://mail.python.org/mailman/listinfo/python-list

[issue43037] Importlib reload by module name (String)

[Stefan Mosoi]

Stefan Mosoi  added the comment:

I understand. 
I was using to reload some classes that might have changed/added (updates and 
stuff) without having to reload the hole project. There might be some other 
ways (i found this, and didn't keep researching after).
The documentation didn't warned / informed about real live usage or intentions 
for this to exist(like in REPL).

I will continue using this as i use it now and i fully accept the consequences. 

Thank you for the time and answers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42973] argparse: mixing optional and positional arguments... not again

[Glenn Linderman]

Glenn Linderman  added the comment:

Yes I think this is a useful enabling step toward enhanced functionality, as is.

But I think the learning curve to achieve the enhanced functionality is a bit 
high for most people, as it requires too much knowledge of argparse internals, 
so I really look forward to your followon work on 'capture' actions. which will 
be an enhancement that will not require internals knowledge, and will resolve a 
day-one deficiency in argparse in a nicely backward-compatible manner.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[C W]

I'm not expert in Python, but I sure tried many IDEs to kick off Python
programming.

I started with PyCharm, but I had a problem with it constantly scanning the
background, even after I turned that feature off.

My favorite (I'm using now) is VS Code with Python extension, it's very
light. Recently also started background scanning, but that's generally done
in 30 seconds.

On Wed, Jan 27, 2021 at 1:51 PM Michał Jaworski  wrote:

> PyCharm has all these debugging capabilities and there is a community
> edition that you can use for free. If you earn for the living with Python
> it is worth investing in professional edition though.
>
> Michał Jaworski
>
> > Wiadomość napisana przez flaskee via Python-list 
> w dniu 27.01.2021, o godz. 19:32:
> >
> > 
> > While print() is groovy and all,
> > if anyone runs across a non-pdb python debugger (standalone or IDE-based)
> > please let me know.
> >
> > I too was blessed with IDE-based debugging (in the 90's!)
> > * where you can set break point(s);
> > * have the program stop right before a suspected failure point;
> > * check the contents of ALL variables, and choose whether to restart;
> > * or skip a few lines before restarting;
> > * or change a variable (hot, move back a few lines and restart, etc.
> > * Some, would even let you alter the code a bit before restarting.
> >
> > I too, miss this.
> >
> >
> > Hopefully I did not miss someone mentioning
> > such a python tool in the prior thread.
> >
> > Thanks!
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

How to accept argparse.log_level parameter and propagate its value to all other modules

[Zoran]

In the same folder I have three files: main.py, app_logger.py and mymodule.py 
with their contents:

# main.py
import argparse
import app_logger
import mymodule
import logging

logger = app_logger.get_logger(__name__)

def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Script for executing data 
quality checks.")
parser.add_argument("--log-level", default='info', choices=['notset', 
'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
args = parser.parse_args()

logger.setLevel(eval(f"logging.{args.log_level.upper()}"))

log_some_messages()
mymodule.log_some_messages()


#mymodule.py
import logging

from script_logging import app_logger

logger = app_logger.get_logger(__name__)


def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')


#app_logger.py
import logging

_log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - 
(%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
_log_level = logging.DEBUG

def get_stream_handler():
stream_handler = logging.StreamHandler()
stream_handler.setLevel(_log_level)
stream_handler.setFormatter(logging.Formatter(_log_format))
return stream_handler

def get_logger(name):
logger = logging.getLogger(name)
logger.setLevel(_log_level)
logger.addHandler(get_stream_handler())
return logger

When I run it with main.py --log-level debug I get the following output:

2021-01-25 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) 
- __main__ - debug message
2021-01-25 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - 
__main__ - info message
2021-01-25 13:21:01,152 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:21:01,152 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:21:01,152 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:21:01,152 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

which is OK. If I change --log-level parameter to 'warning', than output is the 
following:

2021-01-25 13:22:12,760 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:22:12,760 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:22:12,761 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:22:12,761 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

As you can see, in main.py log level is set correctly, but in mymodule.py it is 
not.

How to accept argparse.log_level parameter in if __name__ == '__main__': and 
propagate its value to all other modules?
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43037] Importlib reload by module name (String)

[Brett Cannon]

Brett Cannon  added the comment:

Thanks for the idea, Stefan, but I'm going to close this as something we don't 
want to do. `importlib.reload()` purposefully takes a module object as that's 
what is going to get mutated/changed and it must already exist. The other 
importlib functions take a string because the module might not even exist yet.

And as Serhiy said, reloading is a bit dangerous and shouldn't be taken 
lightly. It primarily exists to reload a module when you're working in the REPL 
and editing a file live, not for anything fancy during execution of production 
code. So keeping it squarely targeted the REPL case and making it a bit harder 
for other cases is a good thing in my opinion.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2824] zipfile to handle duplicate files in archive

[Jerry Heiselman]

Jerry Heiselman  added the comment:

Further, some tools like zipgrep, seem to iterate over the toc in the zipfile 
and end up running the grep part once per entry leading to some duplication of 
data returned without it being obvious that there wasn't actually duplicate 
data in the zip archive.

Interestingly, the zipfile module does not expose the duplicate toc entries 
using 'python -m zipfile -l ', so it's hiding the issue itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2824] zipfile to handle duplicate files in archive

[Jerry Heiselman]

Jerry Heiselman  added the comment:

This just bit us too. I don't feel like a warning is enough. In our case, we 
want to prevent it from happening to begin with so that a developer who isn't 
expecting this doesn't have to know ahead of time to check for a file.

Once a duplicate is in the zip file, it is maintained in the listing of files, 
but deleting one of the entries using Windows explorer results in both entries 
being deleted which can (and has) led to a loss of data. So preventing it from 
happening in the first place seems more prudent. Of course, allow an override 
with a parameter to .write() perhaps.

--
nosy: +jheiselman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: IDE tools to debug in Python?

[Michał Jaworski]

PyCharm has all these debugging capabilities and there is a community edition 
that you can use for free. If you earn for the living with Python it is worth 
investing in professional edition though. 

Michał Jaworski

> Wiadomość napisana przez flaskee via Python-list  w 
> dniu 27.01.2021, o godz. 19:32:
> 
> 
> While print() is groovy and all,
> if anyone runs across a non-pdb python debugger (standalone or IDE-based)
> please let me know.
> 
> I too was blessed with IDE-based debugging (in the 90's!)
> * where you can set break point(s);
> * have the program stop right before a suspected failure point;
> * check the contents of ALL variables, and choose whether to restart;
> * or skip a few lines before restarting;
> * or change a variable (hot, move back a few lines and restart, etc.
> * Some, would even let you alter the code a bit before restarting.
> 
> I too, miss this.
> 
> 
> Hopefully I did not miss someone mentioning
> such a python tool in the prior thread.
> 
> Thanks!
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue22401] argparse: 'resolve' conflict handler damages the actions of the parent parser

[Kuang-che Wu]

Change by Kuang-che Wu :


--
nosy: +kcwu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.

[C W]

For learning purposes, here's the files:
https://www.dropbox.com/sh/a3iy40rcvib4uvj/AAADmlM2i6NquWC1SV0nZfnDa?dl=0

Yes, you are correct about "employee" and "person" discrepancies. For now,
the list comprehension is where I get stuck.

I'd like to know how the experts on here are approaching and debugging
this.

Bonus if no debugger or breakpoint. Just the good ol' run the function and
evaluate/print output for problems.

Thanks so much,

Mike

On Wed, Jan 27, 2021 at 10:53 AM Michael Torrie  wrote:

> On 1/26/21 10:19 PM, C W wrote:
> > Traceback (most recent call last):
> >File "/Users/Mike/Documents/Mike/main.py", line 95, in 
> >   main()
> >File "/Users/Mike/Documents/Mike/main.py", line 86, in main
> >   args = get_feed()
> >File "/Users/Mike/DocumentsMike/main.py", line 32, in get_feed
> >   result = [PERSONatabase.get_person(raw_person) for raw_neo in
> > raw_objects]
> >File "/Users/Mike/Documents/Mike/main.py", line 32, in 
> >   result = [NEODatabase.get_person(raw_person) for raw_neo in
> > raw_objects]
> >File "/Users/Mike/Documents/Mike/database.py", line 24, in get_person
> >   return PERSONDatabase(person['created_at'],
> > KeyError: 'created_at'
>
> The actual error is the last part, which is a KeyError on line 24.  A
> key error usually is from a dictionary-like object and it means the
> requested key is not found in that object.  In other words, this person
> object has no "created_at" key.  Hope that makes sense.
>
> I do not know why the code you posted refers to "employee" but the
> traceback refers to "person."
>
> In any case the trace back shows you what called what until the error
> occurred. You can trace the execution of the code simply by following
> it.  main() called get_feed() which set up a list comprehension, which
> calls get_person() which is where the error is occurring.  I'm not
> following the list comprehension stuff; I don't know why python is first
> referring to PERSONatabase and then refers to NEODatabase.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43042] tutorial ambiguous about creation of local symbol table for recursive calls

[Jesse Silverman]

New submission from Jesse Silverman :

In section 4.6 of the tutorial, we find:

"When a function calls another function, a new local symbol table is created 
for that call."

Now, perhaps because we were just looking at a function that people will often 
ask you to write both recursively and non-recursively and then ask which one 
you would use and why...I was thinking "Wait -- when a function calls itself 
recursively, obviously they need a new local symbol table or local variables 
won't work??"

I may be confused, in which case the doc is fine and I personally need to 
better understand the moral equivalent of activation record / stack frame in 
Python.

If I am not, given that recursion isn't much more computer science oriented or 
obscure than Fibonacci sequences, could it not be worth the investment of a few 
extra words as:

"When a function calls another function, or calls itself recursively, a new 
local symbol table is created for that call."

--
assignee: docs@python
components: Documentation
messages: 385800
nosy: docs@python, jessevsilverman
priority: normal
severity: normal
status: open
title: tutorial ambiguous about creation of local symbol table for recursive 
calls
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

IDE tools to debug in Python?

[flaskee via Python-list]

While print() is groovy and all,
if anyone runs across a non-pdb python debugger (standalone or IDE-based)
please let me know.

I too was blessed with IDE-based debugging (in the 90's!)
 * where you can set break point(s);
 * have the program stop right before a suspected failure point;
 * check the contents of ALL variables, and choose whether to restart;
 * or skip a few lines before restarting;
 * or change a variable (hot, move back a few lines and restart, etc.
 * Some, would even let you alter the code a bit before restarting.

I too, miss this.


Hopefully I did not miss someone mentioning
such a python tool in the prior thread.

Thanks!

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> It might also make sense to build new tokenize.py apis avoiding the 
> `readline()` api -- I always found it painful to work with

Then we would need to maintain the old Python APIs + the new ones using the 
module? What you are proposing seems more than just speeding up tokenize.py 
re-using the existing c code

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

asyncio: Gather over growing list of tasks

[Justin Paston-Cooper]

Hello,

Let's say I have a loop where at each iteration, a task which makes an
HTTP request to some endpoint is generated, and then the loop sleeps
for n seconds. These tasks may throw exceptions.

I can call asyncio.gather on these tasks once the loop has finished,
but I actually want to stop the loop immediately if an exception is
thrown.

Is there a solution that would allow me to stop this loop as soon as
an exception is thrown by any of these request tasks?

Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

I haven't looked into or thought about that yet, it might not be possible

It might also make sense to build new tokenize.py apis avoiding the 
`readline()` api -- I always found it painful to work with

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Either works for me, would you be able to point me to the starting bits as to 
> how `_ast` becomes builtin?


https://github.com/python/cpython/blob/master/Python/Python-ast.c#L10075-L10079 

and 

https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/PC/config.c#L84

But before that I have some questions. For example: How do you plan to 
implement the readline() interface that tokenize.py uses in the c-module 
without modifying tokenize.c?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43036] TOS-behaviour documentation is inconsistent

[Xavier Morel]

Change by Xavier Morel :


--
title: TOS-behaviour documentation is -> TOS-behaviour documentation is 
inconsistent

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

Either works for me, would you be able to point me to the starting bits as to 
how `_ast` becomes builtin?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> private api sounds fine too -- I thought it was necessary to implement the 
> module (as it needs external linkage) but if it isn't then even better

We can make it builtin the same way we do for the _ast module, or we can have a 
new module under Modules (exposing the symbols in the dynamic table) **but** 
making them private (and not documented), which explicitly goes against what 
this issue proposes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

private api sounds fine too -- I thought it was necessary to implement the 
module (as it needs external linkage) but if it isn't then even better

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

For reimplementing Lib/tokenize.py we don't need to publicly expose anything in 
the C-API. We can have a private _tokenize module with uses whatever you need 
and then you use that _tokenize module in the tokenize.py file to reimplement 
the exact Python API that the module exposes.

Publicly exposing the headers or APIs opens new boxes of potential problems: 
ABI stability, changes in the signatures, changes in the structs. Our 
experience so far with other parts is that almost always is painful to add 
optimization to internal functions that are partially exposed, so I am still 
not convinced offering public C-APIs for the builtin tokenizer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

you already have that right now because the `tokenize` module is exposed. 
(except that every change to the tokenization requires it to be implemented 
once in C and once in python)

it's much more frustrating when the two differ as well

I don't think all the internals of the C tokenization need to be exposed, my 
main goals would be:

- expose enough information to reimplement Lib/tokenize.py
- replace Lib/tokenize.py with the C tokenizer

and the reasons would be:

- eliminate the (potential) drift and complexity between the two
- get a fast tokenizer


Unlike the AST, the tokenization changes much less frequently (last major 
addition I can remember is the `@` operator


We can hide almost all of the details of the tokenization behind an opaque 
struct and getter functions

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I assumed, but I don't feel confortable exposing the built-in one.

As an example of the situation, I want to avoid: every time we change anything 
in the AST because of internal details we have many complains and pressure from 
tool authors because they need to add branches or because it makes life more 
difficult for them it and I absolutely want to avoid more of that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com