Re: script question

[Peter Otten]

pyt...@bdurham.com wrote:

> Peter,
> 
>> Another eval-free variant:
>> 
>> [x() for x in vars().values() if hasattr(x, "_included")]
>> 
>> If you use getattr(x, "_included", False) instead of hasattr()
>> you can "un-include" functions with ...
> 
> YES! That's what I was struggling to do with my in-elegant use of
> eval(), eg. replacing eval() to expand dir() strings with a dictionary
> of local objects.
> 
> Question: why vars() vs. globals()? My tests in IDLE (Python 2.6) show
> vars() and globals() returning the same items.

Indeed, it doesn't matter here.

I chose vars() because it uses the same logic as dir(), i. e. vars().keys()
should give you the same names as dir(). Note that both vars() and dir()
give you the same namespace as locals() which just happens to be the same
as globals() on the module level. Inside a function you have to use
globals() or vars(module) to access the module's global namespace.

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[python]

Peter,

> Another eval-free variant:
> 
> [x() for x in vars().values() if hasattr(x, "_included")]
> 
> If you use getattr(x, "_included", False) instead of hasattr()
> you can "un-include" functions with ...

YES! That's what I was struggling to do with my in-elegant use of
eval(), eg. replacing eval() to expand dir() strings with a dictionary
of local objects.

Question: why vars() vs. globals()? My tests in IDLE (Python 2.6) show
vars() and globals() returning the same items.

Thanks,
Malcolm
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Peter Otten]

pyt...@bdurham.com wrote:

> For completeness, here is the corrected version of my original code for
> the archives:
> 
> [ eval(x)() for x in dir() if hasattr( eval(x), '_included') ]

Another eval-free variant:

[x() for x in vars().values() if hasattr(x, "_included")]

If you use getattr(x, "_included", False) instead of hasattr() you
can "un-include" functions with 

f._included = False 

instead of 

del f._included

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[python]

Scott,

Thank you for your example - very clean.

For completeness, here is the corrected version of my original code for
the archives:

[ eval(x)() for x in dir() if hasattr( eval(x), '_included') ]

Regards,
Malcolm
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Arnaud Delobelle]

"D'Arcy J.M. Cain"  writes:

> On 17 Apr 2009 07:03:18 -0700
> a...@pythoncraft.com (Aahz) wrote:
>> Go to all that trouble, you might as well make it easier:
>> 
>> for func in funclist:
>> func()
>
> And if you need the return values:
>
> retlist = [func() for func in funclist]

And if you don't want to be Python 3 compliant:

retlist = map(apply, funclist)

-- 
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Scott David Daniels]

pyt...@bdurham.com wrote:

Scott,

Newbie question (and I'm not the OP): What are your thoughts on having
your decorator add an attribute to the functions vs. placing the
functions in a global variable?

def _included(f):
f._included = True
return f

I tried experimenting with this technique, but could not find a way to
use dir() to discover all my functions with an _included attribute.

The following convoluted comprehension works to execute functions with
an _included attribute when a list of function names is available
However, I can't figure out a way to replace my list of hard-coded
function names with the output from the dir() command.


Here's how I'd do it:

def call_all(module):
for name in dir(module):
contents = getattr(module, name)
if hasattr(contents, '_included'):
yield contents()

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[python]

Scott,

Newbie question (and I'm not the OP): What are your thoughts on having
your decorator add an attribute to the functions vs. placing the
functions in a global variable?

def _included(f):
f._included = True
return f

I tried experimenting with this technique, but could not find a way to
use dir() to discover all my functions with an _included attribute.

The following convoluted comprehension works to execute functions with
an _included attribute when a list of function names is available.

[ eval(x)() for x in [ 'func01', 'func02', 'func03' ] if getattr(
eval(x), '_included', False ) ]

However, I can't figure out a way to replace my list of hard-coded
function names with the output from the dir() command.

Thank you for your thoughts,

Malcolm
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Scott David Daniels]

gitulyar wrote:

On Apr 17, 5:23 pm, Scott David Daniels  wrote:

Marco Mariani wrote:

Piet van Oostrum wrote:

funclist = [func01, func02, func03, ... ]
for i in range(1,n):
funclist[i]()
Or myscript.funclist[i]() from another module.

...
For example, you could do it like:
 funclist = []
 def _included(function):
 funclist.append(function)
 return function

 @_included
 def func01():
 ...
 def call_all():
 for function in funclist:
 function()

--Scott David Daniels
scott.dani...@acm.org


This code does the same as '__all__' should, but anyway it's
the ugly way, because you have to edit '__all__' every time
you add/remove new functions to your module and if '__all__'
is absent your code won't work.

As for me the best solution is to use 'dir()':

for func in dir(myscript):
if match(func):
getattr(myscript, func)()


Well, I prefer that the names of the functions not matter, (that
renaming a function not change the behavior of a program).  And if
you are not going by name, how do you refactor the functions that
you do want to call (and lift common behavior to a function).
It really does come down to style, and I like to avoid using
function names as the way to discover functions.  I put up with
the naming convention for testing, but I don't like it.  The only
excuse I see is that there are often a huge number of tests, and
there _is_ a convention for such names as defined by the xUnit style.
If we were starting from scratch now, I'd argue for a @test decorator.

--Scott David Daniels
scott.dani...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[D'Arcy J.M. Cain]

On 17 Apr 2009 07:03:18 -0700
a...@pythoncraft.com (Aahz) wrote:
> Go to all that trouble, you might as well make it easier:
> 
> for func in funclist:
> func()

And if you need the return values:

retlist = [func() for func in funclist]

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Peter Pearson]

On 17 Apr 2009 07:03:18 -0700, Aahz  wrote:
> In article , Piet van Oostrum wrote:
>>
>>funclist = [func01, func02, func03, ... ]
>>for i in range(1,n):
>>funclist[i]()
>
> Go to all that trouble, you might as well make it easier:
>
> for func in funclist:
> func()

Yes.  Especially because func01 gets called, this way.

-- 
To email me, substitute nowhere->spamcop, invalid->net.
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[gitulyar]

On Apr 17, 5:23 pm, Scott David Daniels  wrote:
> Marco Mariani wrote:
> > Piet van Oostrum wrote:
>
> >> funclist = [func01, func02, func03, ... ]
> >> for i in range(1,n):
> >>     funclist[i]()
>
> >> Or myscript.funclist[i]() from another module.
>
> > Ehm, calling a bazillion things in the right order should be a
> > responsibility of the myscript module anyway.
>
> For example, you could do it like:
>
> myscript.py:
>      funclist = []
>
>      def _included(function):
>          funclist.append(function)
>          return function
>
>     �...@_included
>      def func01():
>          ...
>
>     �...@_included
>      def func02():
>          ...
>
>      def call_all():
>          for function in funclist:
>              function()
>
> --Scott David Daniels
> scott.dani...@acm.org

This code does the same as '__all__' should, but anyway it's
the ugly way, because you have to edit '__all__' every time
you add/remove new functions to your module and if '__all__'
is absent your code won't work.

As for me the best solution is to use 'dir()':

for func in dir(myscript):
if match(func):
getattr(myscript, func)()
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Scott David Daniels]

Marco Mariani wrote:

Piet van Oostrum wrote:


funclist = [func01, func02, func03, ... ]
for i in range(1,n):
funclist[i]()

Or myscript.funclist[i]() from another module.


Ehm, calling a bazillion things in the right order should be a 
responsibility of the myscript module anyway.



For example, you could do it like:

myscript.py:
funclist = []

def _included(function):
funclist.append(function)
return function

@_included
def func01():
...

@_included
def func02():
...

def call_all():
for function in funclist:
function()

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Aahz]

In article , Piet van Oostrum   wrote:
>
>Others have suggested getattr. I think a cleaner (more pythonic) way
>would be: 
>
>funclist = [func01, func02, func03, ... ]
>for i in range(1,n):
>funclist[i]()

Go to all that trouble, you might as well make it easier:

for func in funclist:
func()
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Marco Mariani]

Piet van Oostrum wrote:


funclist = [func01, func02, func03, ... ]
for i in range(1,n):
funclist[i]()

Or myscript.funclist[i]() from another module.


Ehm, calling a bazillion things in the right order should be a 
responsibility of the myscript module anyway.


--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Piet van Oostrum]

> "Stefano"  (S) wrote:

>S> I have a script like this
>S> myscript.py

>S>def func01()
>S>def func02()
>S>def func03()
>S>
>S>def funcnn()

>S> How can i execute my func in the code ?

>S> import myscript
>S> for i in range(1,n):
>S>myscript.func??

Others have suggested getattr. I think a cleaner (more pythonic) way
would be: 

funclist = [func01, func02, func03, ... ]
for i in range(1,n):
funclist[i]()

Or myscript.funclist[i]() from another module.
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[alex23]

On Apr 17, 5:00 pm, "Stefano"  wrote:
> How can i execute my func in the code ?
>
> import myscript
> for i in range(1,n):
>     myscript.func??


for i in range(1,n):
getattr(myscript, 'func%d' % i)()
--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Peter Otten]

Stefano wrote:

> I have a script like this
> 
> myscript.py
> 
> def func01()
> def func02()
> def func03()
> 
> def funcnn()
> 
> How can i execute my func in the code ?
> 
> import myscript
> for i in range(1,n):
  getattr(myscript, "func%02d" % i)()

--
http://mail.python.org/mailman/listinfo/python-list

Re: script question

[Chris Rebert]

On Fri, Apr 17, 2009 at 12:00 AM, Stefano  wrote:
> I have a script like this
>
> myscript.py
>
>   def func01()
>   def func02()
>   def func03()
>   
>   def funcnn()
>
> How can i execute my func in the code ?
>
> import myscript
> for i in range(1,n):
>   myscript.func??
getattr(myscript, "func"+str(n))() #disregarding zero-padding

Cheers,
Chris
-- 
I have a blog:
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

script question

[Stefano]

I have a script like this

myscript.py

   def func01()
   def func02()
   def func03()
   
   def funcnn()

How can i execute my func in the code ?

import myscript
for i in range(1,n):
   myscript.func??

many thanks
stefano
--
http://mail.python.org/mailman/listinfo/python-list

Re: execute python script question

[Piet van Oostrum]

> Gabriel Rossetti <[EMAIL PROTECTED]> (GR) wrote:

>GR> not a stupid question, I think that may be it. I tried setting PYTHONPATH
>GR> like Sam suggested and it worked, but I was unable to do it programmically.
>GR> I tried putting it in the __init__.py file like a web post suggested but it
>GR> wasn't run until after I set PYTHONPATH, and once that was done there is no
>GR> need (that I can see anyways) to set it in __init__.py.

The __init__.py is executed during your import statement, thus it is too
late to find MyPackage. You will have to change the sys.path before the
import, e.g. in your main program. Or do what is usually done: put
MyPackage in the site-packages directory.
-- 
Piet van Oostrum <[EMAIL PROTECTED]>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: [EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: execute python script question

[Gabriel Rossetti]

Michael Wieher wrote:
> stupid question: you have myPackage somewhere on sys.path?
>
> I mean, module1.py only knows it lives in a directory, it doesn't know 
> anything about anything above it.
>
>
>
> 2008/3/10, Gabriel Rossetti <[EMAIL PROTECTED] 
> >:
>
> Hello,
>
> I have been developping something in python that has the following
> hierarchy :
>
> project/src/myPackage/
> project/src/myPackage/__init__.py
> project/src/myPackage/module1.py
> project/src/myPackage/module2.py
> project/src/myPackage/test/
> project/src/myPackage/test/__init__.py
> project/src/myPackage/test/test_module1.py
> project/src/myPackage/test/test_module2.py
> project/src/myPackage/mySubPackage/__init__.py
> project/src/myPackage/mySubPackage/module1.py
> project/src/myPackage/mySubPackage/test/
> project/src/myPackage/mySubPackage/test/__init__.py
> project/src/myPackage/mySubPackage/test/module1.py
> ...
>
> up until now, I had been executing my modules from inside
> project/src/myPackage/
> but I realised that that is wrong (while implementing the test suite)
> and that since all my modules had relative imports (if module2 needed
> module1, it would just say : import module1) I changed them to
> myPackage.module1 for example. Now my test suite is happy, I can say :
> test.sh myPackage.test and it tests everything. The only problem
> now is
> that I can't execute the scripts from inside or outside the myPackage
> dir, I get this :
>
> from outside :
>
> Traceback (most recent call last):
>   File "myPackage/module1.py", line 15, in 
> from myPackage import constants, utils
> ImportError: No module named myPackage
>
> or if from inside it :
>
> Traceback (most recent call last):
>   File "module1.py", line 15, in 
> from myPackage import constants, utils
> ImportError: No module named myPackage
>
> can anybody please help me? I don't think I understood the whole
> package/module thing I think... I think some people do some sort of
> importing in the __init__.py files but I'm not sure this helps in
> this case.
>
> Thanks,
> Gabriel
>
>
Hello Michael,

not a stupid question, I think that may be it. I tried setting 
PYTHONPATH like Sam suggested and it worked, but I was unable to do it 
programmically. I tried putting it in the __init__.py file like a web 
post suggested but it wasn't run until after I set PYTHONPATH, and once 
that was done there is no need (that I can see anyways) to set it in 
__init__.py.

Thanks for your help,
Gabriel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: execute python script question

[Gabriel Rossetti]

Sam wrote:
> Hello,
>
> I may misunderstand your problem, but it may be related to the
> execution environment, especially the PYTHONPATH variable. Have a look
> at the following log:
>
> [EMAIL PROTECTED]:/$ pwd
> /
> [EMAIL PROTECTED]:/$ cat -n /tmp/test_import.py
>  1  class A(object):
>  2  def __init__(self):
>  3  self.value = 1
>  4  def show(self):
>  5  print self.value
> [EMAIL PROTECTED]:/$ python
> Python 2.5.1 (r251:54863, Oct  5 2007, 13:50:07)
> [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>   
 from test_import import A
 
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named test_import
>   
 exit()
 
> [EMAIL PROTECTED]:/$ export PYTHONPATH=/tmp
> [EMAIL PROTECTED]:/$ python
> Python 2.5.1 (r251:54863, Oct  5 2007, 13:50:07)
> [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>   
 from test_import import A
 a=A()
 a.show()
 
> 1
>   
>
> ++
>
> Sam
>   
Hello Sam,

Thank you for your reply. I tried that and it works, thanks. I was 
trying to modify the sys.path in __init__.py and it wasn't working.

Gabriel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: execute python script question

[Sam]

Hello,

I may misunderstand your problem, but it may be related to the
execution environment, especially the PYTHONPATH variable. Have a look
at the following log:

[EMAIL PROTECTED]:/$ pwd
/
[EMAIL PROTECTED]:/$ cat -n /tmp/test_import.py
 1  class A(object):
 2  def __init__(self):
 3  self.value = 1
 4  def show(self):
 5  print self.value
[EMAIL PROTECTED]:/$ python
Python 2.5.1 (r251:54863, Oct  5 2007, 13:50:07)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_import import A
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named test_import
>>> exit()
[EMAIL PROTECTED]:/$ export PYTHONPATH=/tmp
[EMAIL PROTECTED]:/$ python
Python 2.5.1 (r251:54863, Oct  5 2007, 13:50:07)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_import import A
>>> a=A()
>>> a.show()
1
>>>

++

Sam
-- 
http://mail.python.org/mailman/listinfo/python-list

execute python script question

[Gabriel Rossetti]

Hello,

I have been developping something in python that has the following 
hierarchy :

project/src/myPackage/
project/src/myPackage/__init__.py
project/src/myPackage/module1.py
project/src/myPackage/module2.py
project/src/myPackage/test/
project/src/myPackage/test/__init__.py
project/src/myPackage/test/test_module1.py
project/src/myPackage/test/test_module2.py
project/src/myPackage/mySubPackage/__init__.py
project/src/myPackage/mySubPackage/module1.py
project/src/myPackage/mySubPackage/test/
project/src/myPackage/mySubPackage/test/__init__.py
project/src/myPackage/mySubPackage/test/module1.py
...

up until now, I had been executing my modules from inside 
project/src/myPackage/
but I realised that that is wrong (while implementing the test suite) 
and that since all my modules had relative imports (if module2 needed 
module1, it would just say : import module1) I changed them to 
myPackage.module1 for example. Now my test suite is happy, I can say : 
test.sh myPackage.test and it tests everything. The only problem now is 
that I can't execute the scripts from inside or outside the myPackage 
dir, I get this :

from outside :

Traceback (most recent call last):
  File "myPackage/module1.py", line 15, in 
from myPackage import constants, utils
ImportError: No module named myPackage

or if from inside it :

Traceback (most recent call last):
  File "module1.py", line 15, in 
from myPackage import constants, utils
ImportError: No module named myPackage

can anybody please help me? I don't think I understood the whole 
package/module thing I think... I think some people do some sort of 
importing in the __init__.py files but I'm not sure this helps in this case.

Thanks,
Gabriel

-- 
www.mydeskfriend.com
PSE - C (EPFL)
1015 Ecublens, Switzerland
Tel: +41 21 601 52 76
Mob: +41 76 442 71 62

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbee: Simple Backend Python Script Question

[Steve Holden]

joe shoemaker wrote:
> I need to create python script that is threaded. So the main program 
> will run in infinite loop and just retrieving messages and putting them 
> in a queue. (Main thread)
> 
> I need child threads from a pool to process the queue. When there is no 
> stuff in the queue, they go to the pool and become available but they 
> don't terminate. This has to be done continuously.
> 
> Main program need to keep putting stuff in the queue, when there are no 
> messages, then it sleeps for short time and check back to see any messages.
> 
> To do this, I guess you don't write joinAll(), so that the main threads 
> just don't wait for the child but goes to work.
> 
> am I right?
> 
Pretty much. The way I usually do this is to start a number of threads 
reading work items of some sort from a Queue.Queue. The thread will 
block automatically if there's nothing on the queue, resuming when 
something appears (and if I want orderly termination I use None as a 
dummy work unit, and the threads terminate when they receive a None, but 
any sentinel value would do).

> Also, child threads (a function that is threaded) will make connecitons 
> to the database. I am planning to use threadpool, so that threads reuse 
> the connections. So do you close the database connection at the end of 
> the function? If not then the connection will be opened forever?
> 
You don't want a thread pool, you want a connection pool, but there's 
little advantage to having one that is only shared between your threads. 
You really need a system-wide connection pool. Alternatively, you might 
find that your database module is thread-safe to the extent that 
different threads can use cursors created on the same connection without 
interference.

If you want each thread to be able to process transactions that are 
invisible to the other threads before they are committed you should 
ensure that you have a sufficient isolation level, which might imply the 
need for a connection-per-thread architecture, in which case you might 
expect some benefit from connection pooling.

regards
  Steve

regards
  Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd   http://www.holdenweb.com
Skype: holdenweb  http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline

-- 
http://mail.python.org/mailman/listinfo/python-list

newbee: Simple Backend Python Script Question

[joe shoemaker]

I need to create python script that is threaded. So the main program will
run in infinite loop and just retrieving messages and putting them in a
queue. (Main thread)

I need child threads from a pool to process the queue. When there is no
stuff in the queue, they go to the pool and become available but they don't
terminate. This has to be done continuously.

Main program need to keep putting stuff in the queue, when there are no
messages, then it sleeps for short time and check back to see any messages.

To do this, I guess you don't write joinAll(), so that the main threads just
don't wait for the child but goes to work.

am I right?

Also, child threads (a function that is threaded) will make connecitons to
the database. I am planning to use threadpool, so that threads reuse the
connections. So do you close the database connection at the end of the
function? If not then the connection will be opened forever?


Joe
-- 
http://mail.python.org/mailman/listinfo/python-list