Rev fbe428c shows the results of recent experiments using pyzo for syntax coloring in Leo.
You could call this an Engineering Notebook post. You could also say that this discussion should be in #365 <https://github.com/leo-editor/leo-editor/issues/365>, but I wanted to make this discussion more public because it contains a major mystery. Maybe you can help. *tl:dr;* Within pyzo, syntax coloring is 10x *faster *than Leo's. When embedded in Leo, the *same *pyzo coloring code is 10x *slower* than Leo's. This is the mother of all performance bugs. Despite several hours of investigation, including profiling with cProfile, it remains (to paraphrase 2001, a Space Odyssey) a total mystery. This is actually very *good *news, for several reasons: 1. I am now certain that #273 <https://github.com/leo-editor/leo-editor/issues/273> can be fixed. We can get rid of the "Big Text" hack! The proof is simple. Not only does pyzo load (and colorize!) big files quickly, but pyzo delays the *display* of big files until the loading and colorizing is complete. There should be no reason (in principle) while the same behavior can't happen in Leo. In fact, when the pyzo code is enabled, Leo displays the new body text only *after *it has been fully loaded and colorized! The only problem is that it takes 11 seconds (on a slow laptop) for @edit leoApp.py to become visible! 2. The performance bug that afflicts pyzo might also afflict Leo's present syntax coloring code. This is speculative, but worth serious investigation. Pyzo loads *and* colorizes text in about the time it takes Leo to call QTextEdit.setPlainText. This is an important clue. *Prototype code* The top-level pyzo switch in leoColorizer.py enables pyzo syntax coloring. When this switch is True, all of Leo's legacy coloring code simply returns. The present pyzo coloring code is intended to be the simplest adaptation possible. I made the minimum changes to the code. These changes mostly involved initing color settings. The hacks all happen when creating a PyzoHighlighter instance. It's unlikely, but not impossible, that such changes affect performance. *The way forward* Both Leo's existing timing traces and cProfile data clearly indicate all 11 seconds of time are spent in the single call to colorizer.setHighlighter in LeoTree.set_body_text_after_select. This is expected. When pyzo coloring is enabled, this results in a single call to Highlighter.rehighlight as expected. Highlighter is the Pyzo highlighter. As I write this I see it should be named PyzoHighlighter within Leo. There are several possible sources of the performance bug to investigate: 1. How many times does PyzoHighlighter.highlightBlock get called? It might be called way too many times in Leo, for some obscure reason. 2. Are the proper QDocument and QTextEdit being used? The present prototyping code uses an instance of pyzo's CodeEditorBase as way of instantiating the pyzo's syntax colorer. Maybe the highlighter must be attached to Leo's actual QTextEdit body widget. The pyzo colorizer uses QDocument to do crucial caching. Two different QTextEdit's or QDocument's might create the performance bug. 3. What Qt events are being generated? Leo's tree and text events are surely different from pyzo's. Does that make a difference. Could they be causing additional redraws? 4. Does background processing somehow mess things up? Leo's present coloring code spits long coloring tasks into pieces. When pyzo is true, all of the legacy coloring code is supposed to be disabled. But perhaps that background code has some pernicious effect. *Summary* Leo's "ecosystem" is very different from pyzo. Clearly, this makes an enormous difference for the pyzo coloring code. Something *outside* the pyzo coloring code is likely the culprit. Finding it could require a lot of sleuthing. I would appreciate your help on this project. If I get stuck, I might even ask Almar Klein for help... Solving the performance mystery is worth any amount of work. There will be many happy repercussions, including closing #273 <https://github.com/leo-editor/leo-editor/issues/273>. However, this work competes for my attention with other very important work. Edward -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+unsubscr...@googlegroups.com. To post to this group, send email to leo-editor@googlegroups.com. Visit this group at https://groups.google.com/group/leo-editor. For more options, visit https://groups.google.com/d/optout.
