[issue32880] IDLE: Fix and update and cleanup pyparse

Change by Cheryl Sabella : -- dependencies: +IDLE: Incorrect signature in call from editor to pyparse.find_good_parse_start ___ Python tracker

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: from idlelib.pyparse import Parser import timeit code='def f():\n' print(timeit.timeit("statement", # for example "p=Parser(4,4)", globals = globals())) statement microseconds Parser.1 # test timeit code

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: As I mentioned before, Parser.set_code strongly suggests that Parser instances were intended to be reused. But you discovered the bug that prevented this, at least for one code result. With that fixed, it should be possible to have one

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: If you [view] a message, there is an unlink button at the bottom for the author (and believe) and tracker gardeners. I did this for the duplicate and incomplete messages. In the future, lets put timing data for an idea on a separate issue

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: Looking at the creation of the instances of pyparse.PyParse() and hyperparser.HyperParser(), I was a little surprised that they (the instances) are local variables to the methods and aren't instance variables. Since they are called fairly

[issue32880] IDLE: Fix and update and cleanup pyparse

Change by Terry J. Reedy : -- Removed message: https://bugs.python.org/msg312739 ___ Python tracker ___

[issue32880] IDLE: Fix and update and cleanup pyparse

Change by Terry J. Reedy : -- Removed message: https://bugs.python.org/msg312733 ___ Python tracker ___

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: For msg312474 regarding replace vs re. Running the 20,000 copy version of the translated text (only translating once gives the following timings: copies: 2 translate time: 5.591020822525024 copies: 2 translate time:

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: I wish I could delete my last message (and duplicate posting of the one before. On the last one, I didn't move my timings. I feel like an idiot. Anyway, there is a difference. ``` start = time.time() for i in range(100): _tran =

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: I also timed this for a `_tran` optimization (instead of looping and doing the `ord`), but it didn't really make a difference. ``` _tran = defaultdict(lambda: 'x') _tran.update({40: 40,# ord('(') 91: 40,# ord('[')

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: For msg312444 on StringTranslatePseudoMapping: I ran a comparison of the current class vs defaultdict. This was my test: ``` with open('/home/cheryl/cpython/Lib/idlelib/pyparse.py') as fd: code = fd.read() copies = 2 code *= copies

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: For msg312444 on StringTranslatePseudoMapping: I ran a comparison of the current class vs defaultdict. This was my test: ``` with open('/home/cheryl/cpython/Lib/idlelib/pyparse.py') as fd: code = fd.read() copies = 2 code *= copies

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: New issue: find_good_parse_start() call in editor.py has the wrong signature. There's actually a few things going on here. Here's the section in editor: ``` if not self.context_use_ps1: for context in

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: Please go ahead, but coordinate. This issue contains miscellaneous pyparse issues that I thought of while reviewing the tests, plus Serhiy's idea. There should be multiple PRs on dependency issues (which might have more than one closely

[issue32880] IDLE: Fix and update and cleanup pyparse

Cheryl Sabella added the comment: Terry, In order to not duplicate effort, are you going to be working on these or is it ok if I look at them? Should all the changes happen in one PR or do you want to break them down? Thanks! --

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: The ()[]{} match manager is in parenmatch.py. It imports Hyperparser, which uses pyparse.Parser. When one types a closer, the corresponding opener is also highlighted and, depending on the configured style, everything in between. Very

[issue32880] IDLE: Fix and update and cleanup pyparse

Serhiy Storchaka added the comment: Could this code be used in IDLE? For example highlighting the closed/open brace when cursor stay on open/closed brace or shortcuts for "go to open/closed brace"? Many editors have such functionality. -- nosy:

[issue32880] IDLE: Fix and update and cleanup pyparse

Change by Terry J. Reedy : -- dependencies: +IDLE pyparse: fix initialization and remove unused code ___ Python tracker ___

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: One sub-issue is to expand the current short doc section (2.1. Automatic indentation) the intended smart indenting. There is nothing about open parameter lists, not about multiline tuple/list/dict displays. It should say (IN CAPS?) that

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: As noted in the test for find_good_parse_start and PR5755 discussion, a single line header on a line by itself in a multiline comment before a multiline header can prevent recognition of the latter. >>> P.set_str("'somethn=ig'\ndef

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: In _study2, let q instead of p be end of last good line and then set p back to beginning. Setting p to end, q to p, and p to start is confusing. -- ___ Python tracker

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: Is str = str.replace('', 'x') str = str.replace('', 'x') str = str.replace('xx', 'x') str = str.replace('xx', 'x') really faster than _squashex = re.compile('x+').sub # top of file

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: In def find_good_parse_start: str, pos = self.str, None Using str as an attribute is ok, but using the bare built-in class name for an instance thereof is annoying. 's' would be better, or even 'sample' (to be analyzed). --

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: StringTranslatePseudoMapping (Mapping) is an awkward and not very descriptive name. It is similar to collections.defaultdict except that it uses a default value rather than a default factory. The latter is so defaults can be mutables.

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: set_str sets self.str and self.study_level. After the first call, attempts to access unset instance parse attributes from other modules will raise AttributeError. Removing the unneeded class setting will just add 2 more names to the list

[issue32880] IDLE: Fix and update and cleanup pyparse

Terry J. Reedy added the comment: Let's consider the todo questions at the end of the class Parser code. --- # XXX - is this used? lastopenbracketpos = None def get_last_open_bracket_pos(self): "Return index of last open bracket or None."

[issue32880] IDLE: Fix and update and cleanup pyparse

New submission from Terry J. Reedy : Pyparse was mostly written in the early 2000s, with the only one code change since 2007 in 2014. #32874 will add tests for pyparse 'as is' (though with some docstring and comment changes). This issue will change pyparse code, and change