[issue37821] IDLE shell uses wrong namespace for completions

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The current behavior is the quick and easy/dirty solution that we have, not 
what we especially want.  #27609 has multiple issues and ideas for completion 
improvements.  I made testing and improving the code priorities before worrying 
about most of the proposed changes.

Here is where we got confused a bit.  The title refers to shell completions.  
The opening post starts with shell completions and then moves to intended 
behavior for editor completions and its dependence on the presence of a shell.  
I disagreed with the latter and responded, not thinking much about shell 
completions.  Lets stick to shell completions, at least initially (and change 
the title if we include editor completions).

Tal: "Perhaps we should differentiate between completions in the shell vs. when 
editing a file."  I agree that we can consider adjusting the list according to 
the setting.  It is easy enough.  Open_completions, executed in the idle 
process, can detect whether the associated text is the shell or an editor and 
pass a (new) flag to fetch_completions, which will pass it to the 
fetch_completions executed in the run process.
 
Tal: "In the shell, having completions for things not actually available is 
weird".  That I can see and agree to. While the editor allows after-completion 
revisions anywhere in the file before running, the user cannot revise Shell 
history and insert an executed import before executing the current statement.  
A doc statement like "Name and attribute completion in the shell is restricted 
to names and objects available after previous shell statements." seems clear 
enough to anyone who reads it.

As near as I can tell, both before and after the #21261 patch, name (tab) 
completion *is* restricted to valid names.  But attribute completion will 
complete for modules such as socket import by run.py even when not in the main 
namespace.  Python in masOS Terminal (bash, I believe) does attribute 
completion (.), but only for user visible names, not, for instance, for 
reprlib.  The proposal is to restrict IDLE's Shell's attribute completion 
similarly.

Tal: "I really dislike the arbitrary availability of only certain modules for 
completion."  It is not completely arbitrary.  We could complete stdlib names 
from a pre-generated list.  But that would add perhaps 400 names to the name 
completion list.  But we can only complete attributes (which I, at least, use 
much more) for modules in sys.modules. A possible improvement mentioned more 
than once is on-demand import, at least of stdlib modules.  If any are shown to 
be a problem, we could make a blacklist.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

Tal Einat  added the comment:

Terry, many thanks for the clarifications!

> The intended behavior is to complete if at all possible.

I can see the point in that.

> What is odd to me is that you seem happy using the more numerous non-user 
> IDLE imports when there is no Shell.

The alternative would just be completing in an empty namespace, which would be 
completely useless.  I do agree that having a shell should increase the 
available modules etc. available for completion.


FWIW I brought this up trying the dict-key completions in the shell with 
`globals()[` and got surprising results.

Perhaps we should differentiate between completions in the shell vs. when 
editing a file.  In the shell, having completions for things not actually 
available is weird IMO.  In the editor it makes more sense.

Or perhaps we should really go the route of issue #18766 and try to make all 
available modules available for completions.  I really dislike the arbitrary 
availability of only certain modules for completion.  That would likely be a 
*huge* undertaking, though, and I don't think it's worth it.  Maybe restrict 
this to just stdlib modules, and/or use lazy (on demand) importing.

Finally, we can also decide that the current behavior is what we want. It's 
there, documented, and normally not very surprising.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

Change by Tal Einat :


--
Removed message: https://bugs.python.org/msg349556

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

Tal Einat  added the comment:

Terry, many thanks for the clarifications!

> The intended behavior is to complete if at all possible.

I can see the point in that.

> What is odd to me is that you seem happy using the more numerous non-user 
> IDLE imports when there is no Shell.

The alternative would just be completing in an empty namespace, which would be 
completely useless.  I do agree that having a shell should increase the 
available modules etc. available for completion.


FWIW I brought this up trying the dict-key completions in the shell with 
`globals()`, and got very surprising results.

Perhaps we should differentiate between completions in the shell vs. when 
editing a file.  In the shell, having completions for things not actually 
available is weird IMO.  In the editor it makes more sense.

Or perhaps we should really go the route of 18766 and try to make all available 
modules available for completions.  I really dislike the arbitrary availability 
of only certain modules for completion.  That would likely be a *huge* 
undertaking, though, and I don't think it's worth it.  Maybe restrict this to 
just stdlib modules, and/or use lazy (on demand) importing.

Finally, we can also decide that the current behavior is what we want. It's 
there, documented, and normally not very surprising.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I think the current behavior is intended.  The IDLE doc Completions subsection, 
written by Kurt Kaiser in help.text and copied to idle.rst in #5066, has this 
paragraph 

Completions are currently limited to those in the namespaces. Names in an 
Editor window which are not via __main__ and sys.modules will not be found.  
Run the module once with your imports to correct this situation. Note that IDLE 
itself places quite a few modules in sys.modules, so much can be found by 
default, e.g. the re module."

The second sentence is garbled but says that things in sys.modules *will* be 
found.  The fourth sentence points out that sys.modules has many things 
imported by IDLE, not by user code, so they can be found *without running one's 
code*.  The example is "import re...re." and re completions pop up.  I noted on 
#18766 that cleaning up run.py made this *feature* less useful.  It appears 
that this issue and your patch are about disable this intended feature.  I 
don't agree.

To put it another way, "If a shell exists, completion should use the namespace 
used for evaluating code in the shell." is not correct.  The intended behavior 
is to complete if at all possible.  #18766 is about expanding 'possible'.

To constantly maximize 'possible', one would run after every name-binding 
statement (an extreme version of the 3rd sentence above).  I sometimes come 
close to that, though more to catch typos  and rerun tests as I go.

You are correct that entering 're.DOTALL' by completion without also entering 
the required corresponding import results in a NameError when running the code 
.  But the same is true if one enters the same 're.DOTALL' by key.  Either way, 
one has entered the attribute where and when wanted; the NameError says "now go 
back up and add the import".

What is odd to me is that you seem happy using the more numerous non-user IDLE 
imports when there is no Shell.  That would mean that users would initially be 
better off killing the shell when starting to edit.  To me, the logic of 
restricting completions when there is a shell would mean no completions when 
there is not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

Tal Einat  added the comment:

See fix in PR GH-15207.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

Change by Tal Einat :


--
keywords: +patch
pull_requests: +14936
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15207

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37821] IDLE shell uses wrong namespace for completions

[Tal Einat]

New submission from Tal Einat :

Currently, when running an IDLE shell with a sub-process, it will allow 
completing attributes of objects not actually in the shell's namespace. For 
example, typing "codecs." will bring up completions for the codecs module's 
attributes, despite that not having being imported. Further, if one uses the 
completion, this results in a NameError exception, since "codecs" isn't 
actually in the namespace.

AFAICT, the intended behavior is as follows:

* If a shell exists, completion should use the namespace used for evaluating 
code in the shell. (Note that this is slightly different when running a shell 
without a sub-process.)
* If no shell exists (only editor windows are open), completion should use the 
global namespace + sys.modules.

--
assignee: terry.reedy
components: IDLE
messages: 349386
nosy: taleinat, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE shell uses wrong namespace for completions
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com