Re: Managing import statements

[Giovanni Bajo]

Shane Hathaway wrote:

> Here's the real problem: maintaining import statements when moving
> sizable blocks of code between modules is hairy and error prone.


You can also evaluate a solution like this:
http://codespeak.net/py/current/doc/misc.html#the-py-std-hook

-- 
Giovanni Bajo


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Kent Johnson]

Jean-Paul Calderone wrote:
> On Sat, 10 Dec 2005 13:40:12 -0500, Kent Johnson <[EMAIL PROTECTED]> 
>> Do any of these tools (PyLint, PyChecker, pyflakes) work with Jython? To
>> do so they would have to work with Python 2.1, primarily...
> 
> Pyflakes will *check* Python 2.1, though you will have to run pyflakes 
> itself using Python 2.3 or newer.

In that case will it freak when it sees imports that aren't valid in 
CPython, like

import java
from javax.swing import JLabel

or is it doing syntactic analysis without trying to load the referenced 
modules?

Hmm, maybe I should just try it...

Thanks,
Kent
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Jean-Paul Calderone]

On Sat, 10 Dec 2005 11:54:47 -0700, Shane Hathaway <[EMAIL PROTECTED]> wrote:
>Jean-Paul Calderone wrote:
>> On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway <[EMAIL PROTECTED]> wrote:
>>>How about PyLint / PyChecker?  Can I configure one of them to tell me
>>>only about missing / extra imports?  Last time I used one of those
>>>tools, it spewed excessively pedantic warnings.  Should I reconsider?
>>
>>
>> I use pyflakes for this: .  The 
>> *only* things it tells me about are modules that are imported but never used 
>> and names that are used but not defined.  It's false positive rate is 
>> something like 1 in 10,000.
>
>That's definitely a good lead.  Thanks.
>
>> This is something I've long wanted to add to pyflakes (or as another feature 
>> of pyflakes/emacs integration).
>
>Is there a community around pyflakes?  If I wanted to contribute to it,
>could I?
>

A bit of one.  Things are pretty quiet (since pyflakes does pretty 
much everything it set out to do, and all the bugs seem to have been 
fixed (holy crap I'm going to regret saying that)), but if you mail
[EMAIL PROTECTED] with questions/comments/patches, or open a 
ticket in the tracker for a fix or enhancement, someone will 
definitely pay attention.

Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Jean-Paul Calderone]

On Sat, 10 Dec 2005 13:40:12 -0500, Kent Johnson <[EMAIL PROTECTED]> wrote:
>Jean-Paul Calderone wrote:
>> On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway
>> <[EMAIL PROTECTED]> wrote:
>>> How about PyLint / PyChecker?  Can I configure one of them to tell me
>>> only about missing / extra imports?  Last time I used one of those
>>> tools, it spewed excessively pedantic warnings.  Should I reconsider?
>>
>>
>> I use pyflakes for this: .
>> The *only* things it tells me about are modules that are imported but
>> never used and names that are used but n
>
>Do any of these tools (PyLint, PyChecker, pyflakes) work with Jython? To
>do so they would have to work with Python 2.1, primarily...

Pyflakes will *check* Python 2.1, though you will have to run pyflakes 
itself using Python 2.3 or newer.

Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Shane Hathaway]

Chris Mellon wrote:
> On 12/10/05, Shane Hathaway <[EMAIL PROTECTED]> wrote:
>>I'm surprised this problem isn't more familiar to the group.  Perhaps
>>some thought I was asking a newbie question.  I'm definitely a newbie in
>>the sum of human knowledge, but at least I've learned some tiny fraction
>>of it that includes Python, DRY, test-first methodology, OOP, design
>>patterns, XP, and other things that are commonly understood by this
>>group.  Let's move beyond that.  I'm looking for ways to gain just a
>>little more productivity, and improving the process of managing imports
>>could be low-hanging fruit.
> 
> It is probably because most people don't regularly switch that much
> code around, or use that many modules. I think the fact that you move
> that much code between modules is probably a code smell in and of
> itself - you're clearly moving and changing responsibilities.
> Refactoring in smaller chunks, less extreme refactoring, correcting
> the imports as you go, and avoiding inline imports without a really
> good reason will probably help you a lot more than a new syntax.

That's an insightful suggestion, but I'm pretty sure I'm doing it right. 
:-)  I fill my mind with the way the code should look and behave when 
it's done, then I go through the code and change everything that doesn't 
match the picture.  As the code moves closer to production and bugs 
become more expensive, I add a step to the process where I perform more 
formal refactoring, but that process takes much longer.  The problems 
caused by the informal process are less expensive than the formal process.

> Yes. Spend an afternoon looking at PyLints options, get rid of the
> stuff you don't like, and use it regularly. PyDev has PyLint
> integration, which is nice.

Ok, thanks.

> I don't think I've ever imported more than a dozen modules into any
> given file. I rarely find it neccesary to move huge chunks of code
> between my modules - the biggest such I did was when I moved from a
> single-file proof of concept to a real modular structure, and that
> only took me an hour or so. If I'd done it modularly to start with it
> would have been fine.

Well, code moves a lot in the immature phases of large applications like 
Zope and distributed systems.

Shane
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Shane Hathaway]

Jean-Paul Calderone wrote:
> On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway <[EMAIL PROTECTED]> wrote:
>>How about PyLint / PyChecker?  Can I configure one of them to tell me
>>only about missing / extra imports?  Last time I used one of those
>>tools, it spewed excessively pedantic warnings.  Should I reconsider?
> 
> 
> I use pyflakes for this: .  The 
> *only* things it tells me about are modules that are imported but never used 
> and names that are used but not defined.  It's false positive rate is 
> something like 1 in 10,000.

That's definitely a good lead.  Thanks.

> This is something I've long wanted to add to pyflakes (or as another feature 
> of pyflakes/emacs integration).

Is there a community around pyflakes?  If I wanted to contribute to it, 
could I?

Shane
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Kent Johnson]

Jean-Paul Calderone wrote:
> On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway 
> <[EMAIL PROTECTED]> wrote:
>> How about PyLint / PyChecker?  Can I configure one of them to tell me
>> only about missing / extra imports?  Last time I used one of those
>> tools, it spewed excessively pedantic warnings.  Should I reconsider?
> 
> 
> I use pyflakes for this: .  
> The *only* things it tells me about are modules that are imported but 
> never used and names that are used but n

Do any of these tools (PyLint, PyChecker, pyflakes) work with Jython? To 
do so they would have to work with Python 2.1, primarily...

Thanks,
Kent
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Jean-Paul Calderone]

On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway <[EMAIL PROTECTED]> wrote:
>Let's talk about the problem I really want help with.  I brought up a
>proposal earlier, but it was only half serious.  I realize Python is too
>sacred to accept such a heretical change. ;-)
>
>Here's the real problem: maintaining import statements when moving
>sizable blocks of code between modules is hairy and error prone.
>
>I move major code sections almost every day.  I'm constantly
>restructuring the code to make it clearer and simpler, to minimize
>duplication, and to meet new requirements.  To give you an idea of the
>size I'm talking about, just today I moved around 600 lines between
>about 8 modules, resulting in a 1400 line diff.  It wasn't just
>cut-n-paste, either: nearly every line I moved needed adjustment to work
>in its new context.
>
>While moving and adjusting the code, I also adjusted the import
>statements.  When I finished, I ran the test suite, and sure enough, I
>had missed some imports.  While the test suite helps a lot, it's
>prohibitively difficult to cover all code in the test suite, and I had

I don't know about this :)

>lingering doubts about the correctness of all those import statements.
>So I examined them some more and found a couple more mistakes.
>Altogether I estimate I spent 20% of my time just examining and fixing
>import statements, and who knows what other imports I missed.
>
>I'm surprised this problem isn't more familiar to the group.  Perhaps
>some thought I was asking a newbie question.  I'm definitely a newbie in
>the sum of human knowledge, but at least I've learned some tiny fraction
>of it that includes Python, DRY, test-first methodology, OOP, design
>patterns, XP, and other things that are commonly understood by this
>group.  Let's move beyond that.  I'm looking for ways to gain just a
>little more productivity, and improving the process of managing imports
>could be low-hanging fruit.
>
>So, how about PyDev?  Does it generate import statements for you?  I've
>never succeeded in configuring PyDev to perform autocompletion, but if
>someone can say it's worth the effort, I'd be willing to spend time
>debugging my PyDev configuration.
>
>How about PyLint / PyChecker?  Can I configure one of them to tell me
>only about missing / extra imports?  Last time I used one of those
>tools, it spewed excessively pedantic warnings.  Should I reconsider?

I use pyflakes for this: .  The 
*only* things it tells me about are modules that are imported but never used 
and names that are used but not defined.  It's false positive rate is something 
like 1 in 10,000.

>
>Is there a tool that simply scans each module and updates the import
>statements, subject to my approval?  Maybe someone has worked on this,
>but I haven't found the right Google incantation to discover it.

This is something I've long wanted to add to pyflakes (or as another feature of 
pyflakes/emacs integration).

Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Managing import statements

[Chris Mellon]

On 12/10/05, Shane Hathaway <[EMAIL PROTECTED]> wrote:
> Let's talk about the problem I really want help with.  I brought up a
> proposal earlier, but it was only half serious.  I realize Python is too
> sacred to accept such a heretical change. ;-)
>
> Here's the real problem: maintaining import statements when moving
> sizable blocks of code between modules is hairy and error prone.
>
> I move major code sections almost every day.  I'm constantly
> restructuring the code to make it clearer and simpler, to minimize
> duplication, and to meet new requirements.  To give you an idea of the
> size I'm talking about, just today I moved around 600 lines between
> about 8 modules, resulting in a 1400 line diff.  It wasn't just
> cut-n-paste, either: nearly every line I moved needed adjustment to work
> in its new context.
>

You might want to invest in refactoring tools, like the ever-popular
Bicycle Repair Man, which can help ensure that you rename things
correctly.

> While moving and adjusting the code, I also adjusted the import
> statements.  When I finished, I ran the test suite, and sure enough, I
> had missed some imports.  While the test suite helps a lot, it's
> prohibitively difficult to cover all code in the test suite, and I had
> lingering doubts about the correctness of all those import statements.
> So I examined them some more and found a couple more mistakes.
> Altogether I estimate I spent 20% of my time just examining and fixing
> import statements, and who knows what other imports I missed.
>

Both PyLint and PyChecker can detect missing imports (as well as
redundant ones) for you.

> I'm surprised this problem isn't more familiar to the group.  Perhaps
> some thought I was asking a newbie question.  I'm definitely a newbie in
> the sum of human knowledge, but at least I've learned some tiny fraction
> of it that includes Python, DRY, test-first methodology, OOP, design
> patterns, XP, and other things that are commonly understood by this
> group.  Let's move beyond that.  I'm looking for ways to gain just a
> little more productivity, and improving the process of managing imports
> could be low-hanging fruit.
>

It is probably because most people don't regularly switch that much
code around, or use that many modules. I think the fact that you move
that much code between modules is probably a code smell in and of
itself - you're clearly moving and changing responsibilities.
Refactoring in smaller chunks, less extreme refactoring, correcting
the imports as you go, and avoiding inline imports without a really
good reason will probably help you a lot more than a new syntax.

> So, how about PyDev?  Does it generate import statements for you?  I've
> never succeeded in configuring PyDev to perform autocompletion, but if
> someone can say it's worth the effort, I'd be willing to spend time
> debugging my PyDev configuration.
>

PyDev will not generate import statements for you.

> How about PyLint / PyChecker?  Can I configure one of them to tell me
> only about missing / extra imports?  Last time I used one of those
> tools, it spewed excessively pedantic warnings.  Should I reconsider?
>

Yes. Spend an afternoon looking at PyLints options, get rid of the
stuff you don't like, and use it regularly. PyDev has PyLint
integration, which is nice.

> Is there a tool that simply scans each module and updates the import
> statements, subject to my approval?  Maybe someone has worked on this,
> but I haven't found the right Google incantation to discover it.
>

Assuming that you keep your global namespace pure, then you can do
this yourself pretty simply. If you don't, there is no way for
anything to know that you want symbol foo from module bar and not from
module baz.

I don't think I've ever imported more than a dozen modules into any
given file. I rarely find it neccesary to move huge chunks of code
between my modules - the biggest such I did was when I moved from a
single-file proof of concept to a real modular structure, and that
only took me an hour or so. If I'd done it modularly to start with it
would have been fine.

> Shane
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list