Re: [Tutor] Modules and variable usage in functions from other files

[Alan Gauld]

On 24/12/15 01:45, Richard Bekenstein wrote:

>from scipy.constants import m_p,G,k
>import numpy as np
>from math import cos, pi, floor
...
>from bol_runge_kutta_evl_nonadaptive import runge_kutta_evl 

>When I run this main.py, I get an error message from the imported
>'runge_kutta_evl' function that numpy is not defined. Why is it that even
>though I have it defined in my script it does not recognize it in my
>function called to the script? 

The function is in the other module. You only imported the names
into your main module, that does not make them visible in the
other modules.

Making the function *name* visible in main (by importing it) does
not make the function *object* to which the name refers part of
main. The function object remains in the original module. So
you need to import all of the names the function uses into
its module too.(**)

> It turns out that even if I then state
>'import numpy as np' in my 'runge_kutta_evl' function, I then another
>error that 'r_range' is not defined. But aren't 'r_range' and 'np'
>supposed to be accesible to all called functions in the main .py?

They are visible in main but they are not visible in the
other modules that main imports. Importing only makes things
visible to the module doing the importing. Python has no
concept of a "main" module as a kind of global namespace,
it treats your main.py just like any other module. It only
exposes the names you import to main.py Every module must
import the names it needs and in that way becomes an
independent, and therefore reusable, piece of code.

BTW In future, please include the full error message in any
posts. Although in this case we could just about guess
what it looked like, its better to see a cut n' paste
version in case there are subtle clues hidden in the message.

(**)The fact you are getting these kinds of errors suggests
you are not testing the individual modules in isolation
from your main program. It's a good habit to (as a minimum!)
import any modules you write into the interpreter and exercise
the functions manually to be sure they work. That would have
revealed that the modules did not have access to all the
names they needed to work. Even better would be to start
using some of the automated test tools such as unittest
or doctest to ensure your modules work consistently after
each change.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules to work with curl.

[Danny Yoo]

On Fri, Jul 18, 2014 at 1:05 PM, Santosh Kumar  wrote:
> its a curl command caliing a http function.
>
> for example:
>
> curl :  GET http:///app/somefunction

Hi Santosh,

Ok, good.  This is some of the information that we wanted to know.

So you are performing HTTP requests to get content, and you also need
to be able to provide username and password for authentication.
Sounds good so far.



> so i have to use this to create a wrapper which can scale better.

This actually doesn't follow from the requirement.  Let me explain.

You're fixated on using the "curl" shell command to do this task.  But
that's not the only way you can get the content of a URL from Python.
For example, Python's standard library includes a URL-downloading
library in urllib (https://docs.python.org/2/library/urllib.html)
which might work just as well as curl.


You're saying that you need to scale better.  In that case, you must
identify what exactly is causing that slowness.

To that end, please respond to Martin Brown's questions in this
thread.  You should try to address them, because I share those
questions too.


If you're just talking about a single HTTP request, for example, then
I don't see an obvious solution to speed up a single download besides
throwing more hardware and networking equipment at the problem.

But if we're talking about many HTTP requests at once, where the
current solution is getting each URL's content in a serial manner,
then perhaps some kind of concurrent solution will be appropriate.
(e.g. http://www.ibm.com/developerworks/aix/library/au-threadingpython/)
 But such a concurrent approach is only effective if you aren't
already saturating your network, and you're connecting to slow
servers, and... etc.  Those are the details that matter, the ones you
need to talk about and describe.

"Slowness" isn't sufficiently descriptive to let us give you technical
advice.  Please say more about the problem by answering Martin's
questions.


Good luck!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules to work with curl.

[Santosh Kumar]

its a curl command caliing a http function.

for example:

curl :  GET http:///app/somefunction

so i have to use this to create a wrapper which can scale better . While i
use the subprocess it taking some time which i feel as bit slow.

so here are the questions again:
1) what are the different ways to run the above statement in a effective
way ?
2) How to improve the performance if any?

Thanks,
santosh



On Fri, Jul 18, 2014 at 12:45 AM, Danny Yoo  wrote:

> On Thu, Jul 17, 2014 at 11:35 AM, Santosh Kumar 
> wrote:
>
> > I am currently working on a project , which has lots of curl commands. I
> am
> > using subprocess.Popen to run these curl commands. But somehow i see its
> bit
> > slow.
> > Is there a way/module by which i can improve the performance of the
> program.
>
>
> If the use of curl really is responsible and worthwhile to optimize,
> then you might consider using pycurl if you really need the full
> features of curl.
>
> http://pycurl.sourceforge.net/
>
> But can you say more what you're doing with curl?  More information
> may let us give better advice.  The core lesson of
> http://www.cs.bell-labs.com/cm/cs/pearls/cto.html is always in my
> mind: we may not actually know the right question yet, so we must be
> careful about our answers, lest we mislead.
>
> I don't think we really understand the problem yet, so tell us more if you
> can.
>
> I would also recommend profiling if you haven't done so already.
> You've pointed out that something is slow, but unless you've actually
> measured what's really responsible, anything could be contributing.
> It may be, for example, that you're being limited by your network
> throughput, in which case trying to make your program go faster will
> not be effective.  You really must measure: otherwise, it's guesswork
> and crossed fingers.
>
> See:  https://docs.python.org/2/library/profile.html for documentation
> on Python profiling.
>



-- 
D. Santosh Kumar
RHCE | SCSA
+91-9703206361


Every task has a unpleasant side .. But you must focus on the end result
you are producing.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules to work with curl.

[Martin A. Brown]

Hi,

 : I am currently working on a project , which has lots of curl 
 : commands. I am using subprocess.Popen to run these curl commands. 
 : But somehow i see its bit slow. Is there a way/module by which i 
 : can improve the performance of the program.

Wow.  Is curl a requirement or do you simply need to speak HTTP?

Some suggestions and questions come to my mind:

  * Is your problem / cost the forking (i.e. how many times are you 
calling curl)?

  * Is the problem unresponsive or sluggish HTTP servers?

  * Have you localized the slowness?  (I use logging data and timing
information to try to figure that out.)

  * How do you define 'slow'?  How many curl commands is 'lots'?  
Like 100 or 100,000?

Suggestions:

  * Profile the code and see where the slowness is.
 
  * Consider python-requests [0] to handle the HTTP fetches, instead 
of calling subprocess.

  * If after that, you discover that your bottleneck is mostly 
network, consider a task parallelizing solution.  One option 
could be multiprocessing [1].  There are doubtless others.

-Martin

 [0] http://docs.python-requests.org/en/latest/
 [1] https://docs.python.org/2/library/multiprocessing.html

--
Martin A. Brown
http://linux-ip.net/
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules to work with curl.

[Danny Yoo]

On Thu, Jul 17, 2014 at 11:35 AM, Santosh Kumar  wrote:

> I am currently working on a project , which has lots of curl commands. I am
> using subprocess.Popen to run these curl commands. But somehow i see its bit
> slow.
> Is there a way/module by which i can improve the performance of the program.


If the use of curl really is responsible and worthwhile to optimize,
then you might consider using pycurl if you really need the full
features of curl.

http://pycurl.sourceforge.net/

But can you say more what you're doing with curl?  More information
may let us give better advice.  The core lesson of
http://www.cs.bell-labs.com/cm/cs/pearls/cto.html is always in my
mind: we may not actually know the right question yet, so we must be
careful about our answers, lest we mislead.

I don't think we really understand the problem yet, so tell us more if you can.

I would also recommend profiling if you haven't done so already.
You've pointed out that something is slow, but unless you've actually
measured what's really responsible, anything could be contributing.
It may be, for example, that you're being limited by your network
throughput, in which case trying to make your program go faster will
not be effective.  You really must measure: otherwise, it's guesswork
and crossed fingers.

See:  https://docs.python.org/2/library/profile.html for documentation
on Python profiling.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules

[eryksun]

On Wed, Mar 27, 2013 at 5:09 PM, Alan Gauld  wrote:
>
> Tkinter is an unusual module in that it requires support for Tcl/Tk to be
> compiled into the interpreter. This is not always the case.

The _tkinter module is a C extension that links to Tcl/Tk. On Debian
Linux, _tkinter (_tkinter.so) is in the package python-tk /
python3-tk. The ImportError raised by "import tkinter" suggests the
required package. Except for Python 3.2 it incorrectly suggests the
2.x package:

ImportError: No module named _tkinter, please install the
python-tk package

IIRC the official Python installer on Windows defaults to installing
tkinter, but it is optional.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules

[Alan Gauld]

On 27/03/13 19:34, Cor Heisterkamp wrote:


My problem starts with the chapter "Introduction to Modules".
The first line in the program is:
  from tkinter import *
and here I'm getting an error:
  ImportError: No module named tkinter


Thats an unfortunate exampole the author has chosen.

Tkinter is an unusual module in that it requires support for Tcl/Tk to 
be compiled into the interpreter. This is not always the case.


So there are 3 possible causes of the error:

1) Your python does not have support for Tcl/Tk built in.
Which version of Python and which OS are you using? If your version does 
not support Tcl you need to either build a version from source or find a 
copy for your OS with Tkinter support. The default downloads for MacOS, 
Windows and most Linuxes have it but the OS installed versions often don't.


2) The tutor is aimed at Python v3 but you are using v2
Which version of Python are you running? What does it say
when you start the Python interpreter >>> prompt?

3) The tutor and you are both on V2 but you mistyped (or it has a typo) 
that says tkinter when it should say Tkinter.


Try typing

import Tkinter


HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules

[Joel Goldstick]

On Wed, Mar 27, 2013 at 4:12 PM, Mark Lawrence wrote:

> On 27/03/2013 19:34, Cor Heisterkamp wrote:
>
>> Hi,
>>
>> I just started with Python and found a course named "Python programming"
>> by Jody S. Gunther.
>> My problem starts with the chapter "Introduction to Modules".
>> The first line in the program is:
>>   from tkinter import *
>> and here I'm getting an error:
>>   ImportError: No module named tkinter
>>
>> What to do?
>>
>> google it and you find:  http://wiki.python.org/moin/TkInter
>>
>
Step 2 - can Tkinter be imported?

Try the correct command for your version at the Python prompt:

>>> import Tkinter # no underscore, uppercase 'T' for versions prior to V3.0

>>> import tkinter # no underscore, lowercase 't' for V3.0 and later


>
> This works for me with Python 3.3 on Windows Vista.  What OS and Python
> version are you using, as I believe that this was tKinter in Python 2?
>
> --
> If you're using GoogleCrap™ please read this http://wiki.python.org/moin/*
> *GoogleGroupsPython .
>
> Mark Lawrence
>
>
> __**_
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules

[Mark Lawrence]

On 27/03/2013 19:34, Cor Heisterkamp wrote:

Hi,

I just started with Python and found a course named "Python programming"
by Jody S. Gunther.
My problem starts with the chapter "Introduction to Modules".
The first line in the program is:
  from tkinter import *
and here I'm getting an error:
  ImportError: No module named tkinter

What to do?




This works for me with Python 3.3 on Windows Vista.  What OS and Python 
version are you using, as I believe that this was tKinter in Python 2?


--
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.


Mark Lawrence

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules

[Danny Yoo]

> I just started with Python and found a course named "Python programming" by
> Jody S. Gunther.
> My problem starts with the chapter "Introduction to Modules".
> The first line in the program is:
>  from tkinter import *
> and here I'm getting an error:
>  ImportError: No module named tkinter


Case matters.  "Tkinter" should be capitalized.


Best of wishes!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Python tutorial by S. Thurlow - opinions please

[Lisi]

On Saturday 20 August 2011 23:53:53 Alan Gauld wrote:
> On 20/08/11 15:25, Lisi wrote:
> > ridiculous.  I think that I understand how to write a basic function, but
> > I can't work out how to save and call it.
[snip]
> There is a full worked example in the Functions and Modules
> topic in my tutorial...

Bless you, Alan.  Not only is your description great, but it caused me to 
start to read your tutorial.

Bingo.  I think I now understand the conceptual problem I had at the core of 
this.  You actually make it clear in your answer in this thread, but I was so 
bogged down that I didn't take it in.

A function is a sub-routine and not a program. 

That's it.  So simple and I have been stuck on this for weeks!

Now to start again!  When, of course, I may get stuck again.  But I think next 
time I am stuck on something apparently trivial, I'll ask sooner.  And the 
descriptions that you and Steven have given me of how to do it make a lot 
more sense than the ones I was struggling with.

Heartfelt thanks to Alan and Steven.

Lisi
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Python tutorial by S. Thurlow - opinions please

[Alan Gauld]

On 20/08/11 15:25, Lisi wrote:


ridiculous.  I think that I understand how to write a basic function, but I
can't work out how to save and call it.


If you literally mean how to write,save and
use a Python function (from within another script)
then I'll have a go:


How to save and run a bash script:
Write your script


Same with Python, create a text file with your favourite editor and save 
it with a .py extension. This is now a Python module that you can import 
into any other script. Lets assume you called it mymodule.py.



save it in the normal manner


Yes, gotta do that with Python too.


chmod to x for everyone you want to be able to execute it.  (E.g. where owner
is root: perhaps 744)


You don't need that for a Python module, it only needs to be readable. 
(But if you do make it executable you can add a shebang line at the top 
and then run it directly from the command prompt. But since you only 
want to access the functions within we won't need to do that.)



Either move the file into a directory on your path, or add the directory that
the file is in to your path.


Either save the file in a directory in your PYTHONPATH or add the 
directory you saved it in to your PYTHONPATH



It will now run.


It can now be imported.

So to use your function, let's call it spam(), in your mymodule.py file:

Create a new python script, lets assume its called myfile.py.
add the line

import mymodule

call the function with

myVariable = mymodule.spam()

Save your new Python script file

Execute your new file from Pyhon with

$ python /full/path/to/myfile.py

There is a full worked example in the Functions and Modules
topic in my tutorial...

HTH

Alan G.
Author of the Learn to Program web site
http://www.alan-g.me.uk/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Python tutorial by S. Thurlow - opinions please

[Lisi]

Thanks, Steven. :-)  I'll get back to this this evening.
Lisi

On Saturday 20 August 2011 16:51:07 Steven D'Aprano wrote:
> Lisi wrote:
> > I have got myself well and truly bogged down.  I need to change the angle
> > from which I am looking.  I only have a short while left for now in which
> > to make some sort of progress with Python.  What do people think of:
> >
> > http://www.sthurlow.com/python/
> >
> > Is it reasonably accurate and therefore useful?
>
> As far as I can tell after a lightning fast read of it (about three
> minutes to skim a few of the lessons), it seems perfectly fine to me.
>
> [...]
>
> > To show you what I mean:
> > How to save and run a bash script:
> > Write your script
> > save it in the normal manner
> > chmod to x for everyone you want to be able to execute it.  (E.g. where
> > owner is root: perhaps 744)
> > Either move the file into a directory on your path, or add the directory
> > that the file is in to your path.
> > It will now run.
> >
> > Can anyone point me to a similar set of basic instructions for Python
> > modules??
>
> More or less exactly the same, except you need a hash-bang line at the
> top of the script so that the shell knows that it is Python and not a
> shell script. So you can add a line like:
>
> #!/usr/bin/python
>
> at the VERY TOP of your script (it MUST be in the first line for the
> shell to recognise it).
>
> (This is exactly the same practice for all scripting languages,
> including Perl, Bash, Ruby, and many more... if the hash-bang line is
> missing, the shell will try to execute the file with "sh".)
>
>
> As an alternative, you can call the script from the shell prompt like this:
>
>
> $ python path/to/my/script.py
>
> (naturally you don't type the $ that is just the shell prompt)
>
>
> Finally, you can put your script somewhere in the PYTHONPATH and then
> call it like this:
>
>
> $ python -m script
>
>
> Note that this way you leave the .py off the end -- you're not telling
> Python to *run* a file, but to *import* a file, which happens to run it
> as a side-effect. I don't recommend you use this technique until you're
> more comfortable with Python. I mention it only for completeness.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Python tutorial by S. Thurlow - opinions please

[Steven D'Aprano]

Lisi wrote:
I have got myself well and truly bogged down.  I need to change the angle from 
which I am looking.  I only have a short while left for now in which to make 
some sort of progress with Python.  What do people think of:


http://www.sthurlow.com/python/

Is it reasonably accurate and therefore useful?


As far as I can tell after a lightning fast read of it (about three 
minutes to skim a few of the lessons), it seems perfectly fine to me.


[...]

To show you what I mean:
How to save and run a bash script:
Write your script
save it in the normal manner
chmod to x for everyone you want to be able to execute it.  (E.g. where owner 
is root: perhaps 744)
Either move the file into a directory on your path, or add the directory that 
the file is in to your path.

It will now run.

Can anyone point me to a similar set of basic instructions for Python 
modules??



More or less exactly the same, except you need a hash-bang line at the 
top of the script so that the shell knows that it is Python and not a 
shell script. So you can add a line like:


#!/usr/bin/python

at the VERY TOP of your script (it MUST be in the first line for the 
shell to recognise it).


(This is exactly the same practice for all scripting languages, 
including Perl, Bash, Ruby, and many more... if the hash-bang line is 
missing, the shell will try to execute the file with "sh".)



As an alternative, you can call the script from the shell prompt like this:


$ python path/to/my/script.py

(naturally you don't type the $ that is just the shell prompt)


Finally, you can put your script somewhere in the PYTHONPATH and then 
call it like this:



$ python -m script


Note that this way you leave the .py off the end -- you're not telling 
Python to *run* a file, but to *import* a file, which happens to run it 
as a side-effect. I don't recommend you use this technique until you're 
more comfortable with Python. I mention it only for completeness.




--
Steven

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Test Suites

[Lie Ryan]

On 12/30/2009 2:11 AM, Stephen Nelson-Smith wrote:

I do quite a lot of programming in Ruby.  When I do so, my code tends
to have the following layout:

/path/to/src/my_project

Inside my_project:

lib/
test/
my_project.rb

my_project.rb uses classes and helper methods in lib

Inside test, I have a test suite that also uses classes and helper
methods in ../lib

This seems like a sensible way to keep tests and other code separate.

In Python I don't know how to do this so I just have all my tests
in the same place as the rest of the code.

a) Is my above way a sensible and pythonic approach?


Yes, python does not enforce how you should structure your directory 
tree (though you will need to put several __init__.py files here and there).



b) If so - how can I do it in Python?


/path/to/src/my_project
-> lib/
   ` __init__.py
   ` somelib.py
-> tests/
   ` __init__.py
   ` test_one.py
` my_project.py
` run_test.py

and you can reference your libraries and tests like so:

# my_project.py
from lib.somelib import SomeClass
import lib.somelib as sl

# run_test.py
import tests
tests.test_one.run_test()


c) If not, is there a better way than having all the tests in the same
place as the rest of the code?



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modules and Test Suites

[Eike Welk]

Hello Stephen!

On Tuesday 29 December 2009, Stephen Nelson-Smith wrote:
> I do quite a lot of programming in Ruby.  When I do so, my code
> tends to have the following layout:
>
> /path/to/src/my_project
>
> Inside my_project:
>
> lib/
> test/
> my_project.rb
>

> b) If so - how can I do it in Python?

I think the test discovery programs "py.test" and "nose" do what you 
want. If I understand you correctly you want something that sets the 
right search path for library modules. 
http://codespeak.net/py/dist/test/
http://somethingaboutorange.com/mrl/projects/nose/0.11.1/

You could also tweak the environment variable "$PYTHONPATH" or the 
variable "sys.path".
See section 6.1.2:
http://docs.python.org/tutorial/modules.html

I use "py.test" and like it. But I keep tests and libraries in the 
same directory. I know however that "py.test" also supports separate 
directories for tests. 

Eike.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor