Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Wed, Dec 13, 2017 at 12:32 PM, Edward K. Ream 
wrote:

​I was going out the door when I wrote my first brief replies.  Several
questions deserve longer answers.​

I don't like too many tracing code to be left inside code.
>>
>
> ​I understand, but ​for now it helps me understand what I *want* to do.
>

​Furthermore, I have been thinking of this project as a way to peer into
existing code. Let's use the term *reporter code* for this.

I'll create the reporter code using the ast.walk + generator pattern or
perhaps a rewritten ShowData class. Using separate reporter code should
eliminate many traces.

I still don't know how you plan to resolve names to types.
>>
>
> ​These are the difficult resolve methods.
>

​It's easy for me to forget that this is the *one and only* important
question!​

In essence, the present code special-cases ivars injected from outside the
class.  They deserve special attention, but that special case is no
substitute for a more general plan.

As I think of this, it seems that the goals of the project are extremely
limited, namely to check assignments (including correspondence of call args
with signature) in *only* those case where naming conventions are used.

As presently understood, inferences can only be as good or detailed as the
data in the default classes dictionaries. Otoh, there are only a few
official ivars (and chains of same) that Leo devs need to know about, so
*maybe* these kinds of limited inferences *between* classes will suffice.

One can imagine almost unlimited inferences *within* each class, but I
really haven't put much thought into this.

The disappointment I suffered a few days ago arises, I think, from the
limited nature of the checks.

​
>
>> Python 3.6 has added support for type annotations.
>>
>
​I didn't make clear how much I dislike Python's type annotations.  They
may be useful in some cases, but imo they destroy the beauty of Python
code. They have all the charm of C++ templates. They are likely to make
refactoring code significantly harder.

You could say that this project arises from the belief that Leo's naming
conventions say almost as much annotations.

Hope this clarifies my present thinking.

I don't have a deep attachment to this project.  It's already improved
existing code, and the recent patterns you shared with us promise to
collapse the complexity of one or more classes.  Other than that, it may be
that I'll abandon this project in just a few days.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Wed, Dec 13, 2017 at 2:03 PM, vitalije  wrote:

> You can start by studying the resolve methods.
>>
>
> Ok, I am studying it and have a question. In resolve_ivar method if
> ivar=='self' it returns ('class', class_name). Shouldn't it return
> ('instance', class_name)?
>

​Yes.  Thank you.​

>
> Little bit further in same method it checks if ivar is among special_names
> ('c', 'p', 'g', 'v', ...) and if it is it returns spec_obj which is
> ('instance', class...). It seems to me that return values for self and c,
> p, g, v ... are not consistent. Either in both cases it should return
> ('instance', ) or in both cases ('class', ...).​
>

> Am I missing something?
>

​No.  You are right.  The present code should use 'instance' everywhere.

Edward​

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

I see you have added doc strings in c/self_assign functions that says they 
yield for each assignment node.

Actually they yield single value if input node is assignment of correct 
type or don't yield at all. They expect to be called once for each node, 
they do not traverse tree.

Perhaps it would be more readable not to yield data, but to return single 
element list or empty list.
Vitalije



-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

>
> You can start by studying the resolve methods.
>

Ok, I am studying it and have a question. In resolve_ivar method if 
ivar=='self' it returns ('class', class_name). Shouldn't it return 
('instance', class_name)?

Little bit further in same method it checks if ivar is among special_names 
('c', 'p', 'g', 'v', ...) and if it is it returns spec_obj which is 
('instance', class...). It seems to me that return values for self and c, 
p, g, v ... are not consistent. Either in both cases it should return 
('instance', ) or in both cases ('class', ...).

Am I missing something?
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Wed, Dec 13, 2017 at 11:50 AM, vitalije  wrote:

I didn't like the Type class. It can be replaced with named or even
> ordinary tuple.
>

​We'll see...
​

> I couldn't find that you have instantiated it anywhere with last two
> optional arguments.
>

​Old cruft.
​

> Also it defines method `__eq__`, but it is never used (or I didn't find
> usage).
>

​The code crashes without it.

I don't like too many tracing code to be left inside code.
>

​I understand, but ​for now it helps me understand what I *want* to do.

I still don't know how you plan to resolve names to types.
>

​These are the difficult resolve methods.
​

> I don't see how it can be relevant for other cases?
>

​That's what I am trying to understand.
​

> Python 3.6 has added support for type annotations.
>

​I am aware of annotations. ​I've written a tool that helps create
annotations automatically.

I've abandoned the idea of static type checking.  However, the rope package
does a pretty good job of that.  It might be useful to add support for rope
to Leo.

If you have a concrete data structure that would allow type checking, and
> that you can share (even if it is hand written for simple code example), I
> would be glad to further develop my prototype for extracting all required
> data from source code in a format that you need.
>

​The present code makes some inferences. You can start by studying the
resolve methods.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

>
> P.S. A nit.  I dislike "ok and" pattern.  For example, I would rewrite 
> is_assign_to this way:


I don't mind your way of rewriting it. I used it just because I didn't know 
in advance all checks that need to be fulfilled, and `ok = ok and ...` 
 gave me opportunity to put trace between any two and see what was the 
reason that some function returned False. Also, at some point I can use `ok 
= ok or `.

I am glad that you liked this approach.  If I knew more precisely what data 
you want to have after second pass, I would go for it.

I didn't like the Type class. It can be replaced with named or even 
ordinary tuple. I couldn't find that you have instantiated it anywhere with 
last two optional arguments. Also it defines method `__eq__`, but it is 
never used (or I didn't find usage). It looks as if you thought it would be 
needed, but until it is really needed it just consumes brain power.

I don't like too many tracing code to be left inside code. It consumes a 
lot brain cycles  to read it and it hides real effective code. For example: 
all methods do_... take as an argument regex match and then again they 
throwaway input argument and compute same match once again. You said that 
patterns are defined where they are used, but it seems to me, that it was 
the case at some earlier stage, and now it is not the case. It is not as 
obvious mostly due to the presence of tracing code. If you turn on all of 
the traces I believe it would produce so much output that it would be 
challenging to read and understand traces. IMHO, traces should be always on 
while debugging one or two functions, but as soon as debugging is over, I 
would delete traces. For me it is much easier to delete and retype them 
even ten times if necessary than to read them all the time. Typing-in code 
is much cheaper than studying it.

I still don't know how you plan to resolve names to types. I see relatively 
straightforward way to check and find the error in given example code, but 
I don't see how it can be relevant for other cases? Python 3.6 has added 
support for type annotations. Maybe that is the best way to define and 
check type conventions. When I ran my code on whole Leo code base I was 
puzzled by the fact that there is just a few places where pattern c. 
= self is used. 
Notes-->@file ../doc/leoAttic.txt-->Unused code-->Static type checking...

As you can see under 'Unused code'. How useful would be leoCheck in the 
end? Hopefully, I am missing something.

If you have a concrete data structure that would allow type checking, and 
that you can share (even if it is hand written for simple code example), I 
would be glad to further develop my prototype for extracting all required 
data from source code in a format that you need. 

Vitalije


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Wed, Dec 13, 2017 at 7:44 AM, vitalije  wrote:

> Here is my attempt to show in code what I couldn't explain with the words.
>

It looks like you forgot these lines in my_check_func:

for x in self_assign(_n):
s_assigns.append(x)​

As you can see, I am studying the code closely.  There are many elegant,
useful ideas.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Wednesday, December 13, 2017 at 7:44:57 AM UTC-6, vitalije wrote:

Here is my attempt to show in code what I couldn't explain with the words. 
>
... 

> current version successfully adds entries to map of known_classes. In 
> particular it adds injected theTagController to the Command class ivars.
>
> Value that my code adds for each of those entries is AST node, containing 
> all relevant information. It can be examined further when resolving. 
>

Many thanks for this excellent work.  The code snippets are great tools. 
I'll use something like this as the basis of the production version.

It may take awhile for me to adjust to this way of thinking, but it is 
obviously better than my previous ast code.

Edward

P.S. A nit.  I dislike "ok and" pattern.  For example, I would rewrite 
is_assign_to this way:

def is_assign_to(node, _id):
'''
returns True if this node is assignment to some attribute of variable 
named _id.

Something like: `_id. = ...`
'''
if isinstance(node, ast.Assign) and len(node.targets) == 1:
arg = node.targets[0]
return (
isinstance(arg, ast.Attribute) and
isinstance(arg.value, ast.Name) and
arg.value.id == _id
)
else:
return False

And I also prefer docstrings to comments ;-)

P.P.S. The new ProjectUtils. leo_core_files method is preferable to the 
leoFiles list.

EKR

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

Here is my attempt to show in code what I couldn't explain with the words. 
Attached to this message you can find Leo document with script node that 
analyzes little bit modified your test example which demonstrates wrong 
method call. My code in the current version successfully adds entries to 
map of known_classes. In particular it adds injected theTagController to 
the Command class ivars.

Value that my code adds for each of those entries is AST node, containing 
all relevant information. It can be examined further when resolving. 

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.


LeoCheck-vitalije.leo
Description: Binary data

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Tuesday, December 12, 2017 at 1:06:50 PM UTC-6, Edward K. Ream wrote:

As you can see from my lack of comments on other matters, this project 
requires my undivided attention.

Recent work:

1. The "kind" switch in checker.check is one of those small things that 
makes a surprisingly big difference.  It replaces several switches, and is 
mostly self documenting.

2. The prototype can report many useful data.  You could call it another 
version of the ShowData class.

3. Early this morning I failed in the first attempt to generalize 
do_assn_to_c.  I'll try again today.

4. It's still unclear how useful this tool will be.  Two details relate to 
this question:

- The code now suppresses the creation of entries for "special" classes, 
those classes for which the use has supplied type information.  The idea is 
that the user has specified the public interfaces for those classes, so 
inferences should use no other data. This suppression also simplifies dumps 
of symbol tables.

- Otoh, the code *must *handle *injected ivars*, that is, ivars that are 
set from outside the class. Handling injected ivars is essential for 
handling the original bug.  Without this kind of knowledge the user would 
have to supply way too much information. Furthermore, reporting injected 
ivars is likely to be useful, if for no other reason that it would suggest 
other knowledge that the user could supply automatically.

5. Recent changes work around parsing errors in the regex's.  There is no 
way that regex can do a full, accurate job of parsing python.

I am considering creating a version of the prototype that uses the ast in a 
more traditional way.  It will be interesting see how big an impact this 
switchover will have. Otoh, the switchover is considerably less important 
than inference-related work, so I may restrain myself a little longer.

6. There is a weird bug in the rewritten ProjectUtils.project_files method. 
When first reloading Leo it can fail to find any files in the 'leo' 
project. As I write this, I see that the present code is brain dead.  For 
leo, it imports leo.core and uses leo.core.__file__ to find the base 
directory.  But Doh, that's just g.app.loadDir!

*Summary*

- assn_to_c should be generalized.  That will allow the prototype to report 
injected ivars.
- The resolve code should be simplified.
- It's nearing time to transition to ast-based code.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Tuesday, December 12, 2017 at 11:58:07 AM UTC-6, Edward K. Ream wrote:

> My previous bad/fatal decisions all related to worrying about details 
instead of focusing on inference.

Lately I've been using the mental trick of trying to make the code *slower* 
than before, by making more inferences or by making the code more general. 
This is a partial antidote to my typical obsession with speed.

In a production version, it might be good to make both the ast node and its 
stringized version available at all times. The ast is required to handle 
nested classes, but the string representation of statements should also be 
useful.  How else to honor user settings?  Well, maybe there are other 
ways. This is an open question.

A few words about generators:

1. Generators can make debugging difficult, because tracing their state 
*always* consumes them. I've asked Guido and other Python guru's about 
this.  list(aGenerator) doesn't always work for debugging.  This was a 
serious problem when trying to find the deep pylint bug.  I couldn't see 
the intermediate generator states!

2. Having said that, using a generator/functional pattern for the resolve* 
methods could be a very good idea.  I see no obstacle to doing so.  This 
might be the most important takeaway from our discussion.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Tuesday, December 12, 2017 at 8:06:15 AM UTC-6, vitalije wrote:

> I was unpleasantly surprised with the complexity of code, and yet you 
wrote: "Otoh, the infrastructure is simple and good."

The infrastructure is similar to, but simpler than, tools such as pyflakes, 
pylint and my previous tools.

I know of no way to avoid traversing ast trees in this kind of problem.

> I believe that it looks simple in your mind while it is still fresh. But 
in mind of a programmer who reads it for the first time it is very hard to 
understand. I guess you would feel the same when you return to this code 
after several months from now.

I doubt it. Compiler-like tools have their own complexity metrics. And 
their own design/coding patterns.

> I was expecting to see something like a function that takes a filename or 
its content as a string and a list of conventions (possible some flags 
too), and returns a generator of all violations in a form (line number, 
line contents, ...).

That would be a most unusual pattern. I've never seen anything like it in 
all the tools I have studied.

There are two tasks facing any inference engine such as pyflakes, pylint or 
check-conventions:

1. Create data structures describing the program being analyzed.
2. Use those data to make inferences.

To understand the inherent complexities, please study the pyflakes and 
pylint code, as I have done. The contrast between the pylint and pyflakes 
is stark:

- Pyflakes is arguably the most elegant code I have ever studied.  I am 
happy to call it a work of genius.  In essence, it manages to get the 
benefits of a two-pass algorithm in a single traversal of the ast. It also 
catches way more practical bugs than pylint.

- In contrast, pylint is unimaginably complicated.  My one claim to fame re 
pylint is that I fixed a bug in pylint 
 that occurred several 
hundred levels down in the call stack.  I used a special purpose debugger 
(a subclass of pdb) to do this.  Despite this brag, I still don't 
understand pylint in detail.

In fact, the check-convention code simulates an ast traversal.  The do_* 
methods correspond directly to actual ast visitors.  So yes, I insist, the 
organization of check-convention is, by the standards of similar tools, 
ordinary and straightforward.

> Instead I have found 7+ classes. 

The ConventionChecker is the only class you need to understand. It contains 
two simple "internal" classes, Type and CCStats.  Oh yes, the code does use 
the simple ProjectUtils class.

Please ignore the other classes.  

For now, ConventionChecker needs to be in leoCheck.py so that other code 
can use imp.reload.  Eventually, I suppose, it should migrate into 
checkerCommands.py.

> Now, when someone new to this code reads it, it is not clear where to 
find what is in self.patterns. 

These patterns are defined just before the do_* methods, where they are 
actually used.

You can find them with cff: patterns.append

> Wouldn't be much simpler to have dispatch definition contains pairs of 
(regex pattern, method) or if kind is really necessary for something else 
then dispatch can have a form {pattern: (method, kind)}.

No.

The infrastructure is, really and truly, straightforward. The 
infrastructure merely ensures that the do_* visitors get called as 
expected. 

> Additionally, we could have a helper function.

I'm not going to comment on this.  The present code works well enough.

The only thing that matters are the resolve_* methods.  They aren't easy, 
and can never can be.  But they are a lot easier than the corresponding 
code in pylint.

> The content of file being checked is parsed into ast node, and then that 
ast node is formatted again to string to be analyzed further.

Yes.  This is a brilliant simplification, if I do say so myself.

> It involves call to leoAst.AstFormatter which uses visitor pattern. It is 
far from clear what is happening. 

Brilliance sometimes is a bit difficult to understand.  

> It took me quite a while to figure that all this code have just a simple 
purpose to clean a bit source code. 

It's way more than that:

1. Each statement now appears on it's own line, simplifying the regex's.
2. Comments have disappeared.
3. Most importantly, using strings makes it possible to apply regex's to 
each statement.  This is crucial for the prototype!

> Wouldn't be much easier to read and understand if those two phases were 
gathered inside one function named `clean_source`: 

No.

> inside start_class method there is also a guard `if self.pass_n==1` which 
means it should also be called only in the first pass.

I've cleaned up this code.  The old start_class method is gone.

> Is class Pass1 used at all?

check-conventions uses only the ConventionChecker and ProjectUtils classes.

> And [Pass1] is very complex class to understand.

It was a previous failed attempt. It never got around to making any type 
analysis at all.

I've left Pass1 in leoCheck.py beca

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Tue, Dec 12, 2017 at 8:06 AM, vitalije  wrote:

Without any wish to degrade your work on this project, I feel strong need
>> to suggest some ideas. Feel free to ignore them if you find them
>> unappealing.
>>
>
​I'm overwhelmed with ideas/tasks on the project now.  I'll respond when
the dust clears.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

I don't know what is wrong with googles code blocks but all the code 
examples in my previous message got somehow wrongly reformatted. 
I will try to edit them.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: ENB: What's next for check-conventions

[vitalije]

>
>
> *Maintain code simplicity!*
>
> Without any wish to degrade your work on this project, I feel strong need 
to suggest some ideas. Feel free to ignore them if you find them 
unappealing.

This morning I checked what you have done so far and I was unpleasantly 
surprised with the complexity of code, and yet you wrote:

> Otoh, the infrastructure is simple and good. Speed is good enough. The new 
> Stats class is much more clever than before.


I believe that it looks simple in your mind while it is still fresh. But in 
mind of a programmer who reads it for the first time it is very hard to 
understand. I guess you would feel the same when you return to this code 
after several months from now.

First of all, before I looked in the code, I was expecting to see something 
like a function that takes a filename or its content as a string and a list 
of conventions (possible some flags too), and returns a generator of all 
violations in a form (line number, line contents, ...). Instead I have 
found 7+ classes. 

Here are some points I collected while exploring leoCheck.py:


   1. ConventionChecker defines class field patterns as empty list and then 
   later throughout the definition of class patterns are appended to this 
   field. Value of this field is used only once in method check_helper 
   def check_helper(self, fn, node, s):
   
   trace = False and self.enable_trace
   
   trace_source = False
   
   self.file_name = fn
   
   dispatch = {
   
   'call': self.do_call,
   
   'class': self.do_class,
   
   'def': self.do_def,
   
   'self.x=': self.do_assn_to_self,
   
   'c.x=': self.do_assn_to_c,
   
   }
   
   s1 = leoAst.AstFormatter().format(node)
   
   for n in (1, 2):
   
   self.enable_trace = n == 2
   
   self.class_name = None
   
   self.pass_n = n
   
   if trace:
   
   print('= pass: %s' % n)
   
   if trace_source and n == 2:
   
   print('= source')
   
   print(s1)
   
   print('- end source')
   
   for n, s in enumerate(g.splitLines(s1)):
   
   self.line_number = n
   
   for kind, pattern in self.patterns:
   
   m = pattern.match(s)
   
   if m:
   
   f = dispatch.get(kind)
   
   f(kind, m, s)
   
   self.start_class()
   
   self.end_program()
   
   
   
   Now, when someone new to this code reads it, it is not clear where to 
   find what is in self.patterns. It is obvious that self.patterns contains 
   pairs (kind, pattern), but it is far from obvious where those pairs come 
   from. Wouldn't be much clearer to explicitly spell all those pairs in `for` 
   statement. First part of each pair `kind` has sole purpose to connect 
   through `dispatch` pattern with the corresponding method. That is another 
   unnecessary indirection that just adds to the complexity. Wouldn't be much 
   simpler to have dispatch definition contains pairs of (regex pattern, 
   method) or if kind is really necessary for something else then dispatch can 
   have a form {pattern: (method, kind)}. Even further it is somewhat 
   misleading to iterate over patterns for each line of source code, when in 
   fact it is possible only one pattern to match a single line. Wouldn't be 
   much more readable to have a sequence of `if elif elif elif elifelse` 
   and get rid of patterns, dispatch and `for` statement. Additionally, we 
   could have a helper function: 
   def check_pattern(line, pat, func):
   m = pat.match(line)
   if m:
   func(m, line)
   
   return True
   
   for n, s in enumerate(g.splitLines(s1)): self.line_number = n 
   check_pattern(s, assn_to_c_pattern, self.do_assn_to_c) or \ 
   check_pattern(s, assn_to_self_pattern, self.do_assn_to_self) or \ 
   check_pattern(s, call_pattern, self.do_call) or \ check_pattern(s, 
   class_pattern, self.do_class) or \ check_pattern(s, def_pattern, 
   self.do_def)
   Imagine this snippet inside check_helper method. Isn't it simpler? More 
   readable? Going a little bit further, methods do_... can be also defined 
   inside check_helper method. Three of them have guard `if self.pass_n == 1` 
   and pass_n is set inside check_helper. Were this three methods functions 
   defined inside check_helper, they would have direct access to pass_n 
   variable. do_class is in fact equivalent with start_class and the only 
   thing start_class do is to set self.class_name and a default dict for 
   self.classes[self.class_name] (it really would be better to use 
   self.classes=collections.defaultdict({'ivars':{}, 'methods':{}})
   class_name ivar is used in other do_... methods which means if 
   start_class was replaced with the simple assignment to a local variable 
   inside check_helper do_ functions could have direct access to it or it 
   could be passed as argument where needed. Given the fact that some of this 
   methods are used only in first pass and do_call is used in both passes it 
   would make sens to get rid of the outer loop also and just have two 
   different functions `do_pass_one` and `do_pass_two` which would 

Re: ENB: What's next for check-conventions

[Edward K. Ream]

On Monday, December 11, 2017 at 10:11:33 AM UTC-6, Edward K. Ream wrote:

> The only thing that matters now is...checking function calls against 
their signatures.

Recent revs do this with straightforward code.  Here are the statistics 
from checking all of Leo's core files:

78 files in 3.39 sec.
assignments: 2973
  calls: 13871
check_signature: 2085
classes: 406
   defs: 5613
resolve: 17035
   resolve_call: 13871
  resolve_chain: 12806
   resolve_ivar: 10944
   sig_infer_ok: 7 # inferred matches with known names
 sig_ok: 1143 # exact match
sig_unknown: 942 # mismatches involving unknown names

The last three stats are for the signature checks.  They are a bit 
underwhelming because not many inferences actually happen.  Otoh, the 
inferences are good enough to catch the original tricky bug.

> I'll also check the contents of symbol tables.

Still on the agenda. I'll also study the unknown signature case a bit 
further.

*Summary*

I feel a bit let down by the results.  It's time for a break.

Otoh, the infrastructure is simple and good. Speed is good enough. The new 
Stats class is much more clever than before. 

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.