[Tutor] Is nesting functions only for data hiding overkill?

[James Hartley]

Yes, nesting functions is valuable & necessary for closures and wrapping
functions for creating properties.  But is nesting, simply for hiding data,
a preferred solution?  I have a number of member functions which are
prefaced with underscores pointing out that they should not be called by
client code.  Is nesting considered Pythonic?

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Is nesting functions only for data hiding overkill?

[James Hartley]

Yes, nesting functions is valuable & necessary for closures and wrapping
functions for creating properties.  But is nesting, simply for hiding data,
a preferred solution?  I have a number of member functions which are
prefaced with underscores pointing out that they should not
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] class functions/staticmethod?

[James Hartley]

I am lacking in understanding of the @staticmethod property.
Explanation(s)/links might be helpful.  I have not found the descriptions
found in the Internet wild to be particularly instructive.  Given the code
below:
=8<--
from collections import namedtuple

class Foo():
Dimensions = namedtuple('Dimensions', ['height', 'width'])
_dimensions = Dimensions(3, 4)

def dimensions():
print('id = {}'.format(id(Foo._dimensions)))
return Foo._dimensions

@staticmethod
def dimensions1():
print('id = {}'.format(id(_dimensions)))
return _dimensions
=8<--
The class method Foo.dimensions() is capable of accessing class members,
but Foo.dimensions1() cannot. What does the @staticmethod decorator really
add?

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Inserting long URL's into comments & docstrings?

[James Hartley]

This should be a slow ball pitch.  Unfortunately, I haven't stumbled across
a reasonable answer yet.

On occasion, I put long URL's into comments/docstrings simply to document
where I found specific information.  However, to be a good disciple of
PEP8, anything which can't fit within 72 characters needs to be split
across multiple lines.  Since a number of you seem to be prolific Python
coders, what opinion do you have about splitting URL's in
comments/docstrings?

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] raising exceptions in constructor code?

[James Hartley]

I ask this having more C++ knowledge than sense.

There is an adage in the halls of everything Stroustrup that one needs to
think about how resource allocation will be unwound if an exception is
thrown.  This gets watered down to the mantra "Don't throw exceptions from
within constructors."  Does this carry over to Python?  I'm trying to
develop a Pythonistic mindset as opposed to carrying over old baggage...

Thanks!

Jim
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Lengthy copyright notices?

[James Hartley]

help(module_name) will place any text in the *first* module-level docstring
into the description section of the help page in Python 3.4.5.  Subsequent
docstrings found at module level are ignored.

I have been using this factoid for placement of a copyright & licensing
notice.  By placing a rather lengthy copyright & license in the code in a
the second module-level docstring, it is prominent within the code, but not
cluttering up help() output.

Two questions.  Is there a more standardized way of including long license
descriptions in code, & is it documented that any other module-level
docstring will be ignored in help() output?

Thanks!

Jim
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] directory structure with tests?

[James Hartley]

It is preferable to sprinkle tests files throughout the directories of a
project, or coalesce all tests in a test directory?

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Beginners Book, Python and PyScripter

[James Stapleton-Cotton]

Thank you Adam.
I am currently going through the book, 'A Byte of Python' - found here -
https://python.swaroopch.com.
On pages 29-30 it writes about 'Using A Source File', creating a file
called '/tmp/py' on a Mac OS using the 'bash-terminal' and running the
saved PyCharm file, 'hello.py'. I type 'mkdir /tmp/py' into the terminal
and press return. This supposedly creates a new folder. I type 'mkdir
/tmp/py' a second time and press return as so:

USERs-MacBook-Pro:~ user$ mkdir /tmp/py

mkdir: /tmp/py: File exists


This shows that the file is indeed created and 'exists'. In the next
instructed step (typing 'python hello.py' into the terminal) there shows an
error as such:

USERs-MacBook-Pro:~ user$ mkdir /tmp/py

mkdir: /tmp/py: File exists

USERs-MacBook-Pro:~ user$ python hello.py
python: can't open file 'hello.py': [Errno 2] No such file or directory

I previously created a python file called 'hello.py' using the PyCharmEdu
editor (as instructed). This file is saved in my 'Documents' in a folder
called 'Pycharm Projects' [I tried changing the location of the file to
'/tmp/py', however this created an error during my 'print' function -
print("hello world") - in the editor, therefore I changed the location back
to the original, 'PycharmProjects'.]

I have found the /tmp/py folder within my 'Users' folder. I dragged the
'hello.py' file into the tmp folder and attempted the above process of
'running' the 'python hello.py' file from the terminal, however the same
result occurred.

Could you please help clarify where I am going wrong in the instructed
process given by the book mentioned above.

Regards

On Tue, Dec 4, 2018 at 9:16 PM Adam Eyring  wrote:

> I haven't gone through many python books, but have been using a copy of
> Automating the Boring Stuff with Python. It covers lists, dictionaries,
> scraping data from websites, etc.
> https://automatetheboringstuff.com/
> The PDF is free.
>
> Adam
>
> On Tue, Dec 4, 2018 at 1:09 PM James Stapleton-Cotton 
> wrote:
>
>> Thank you Mats and Steven. I'm back on track now with a different tutorial
>> which has lead to me to the relevant coding tools.
>> https://wiki.python.org/moin/BeginnersGuide/NonProgrammers
>> https://python.swaroopch.com
>>
>> On Mon, Dec 3, 2018 at 4:04 PM Mats Wichmann  wrote:
>>
>> > On 12/3/18 3:35 AM, James Stapleton-Cotton wrote:
>> > > Hello,
>> > >
>> > > On this page (
>> > >
>> >
>> http://openbookproject.net/thinkcs/python/english3e/way_of_the_program.html
>> > )
>> > > - a book for learning Computer Science using Python - I am directed
>> to (
>> > > http://code.google.com/p/pyscripter) in order to access the
>> appropriate
>> > > program development environment, PyScripter. I have downloaded the
>> latest
>> > > version of Python from Python.org and PyScripter from a site that I am
>> > > directed to from the original site mentioned in the beginning of this
>> > email
>> > > - (https://sourceforge.net/projects/pyscripter/files/), however I
>> can't
>> > > seem to be able to run PyScripter on my Mac.
>> >
>> > it looks like from the project's own description that PyScripter is
>> > intended solely for the Windows environment.
>> >
>> > But there are a TON of other IDE products, free and commercial, out
>> > there, many of which do run on the Mac.
>> >
>> > This may be information overload, but try looking here:
>> >
>> > https://wiki.python.org/moin/IntegratedDevelopmentEnvironments
>> >
>> >
>> > ___
>> > Tutor maillist  -  Tutor@python.org
>> > To unsubscribe or change subscription options:
>> > https://mail.python.org/mailman/listinfo/tutor
>> >
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Beginners Book, Python and PyScripter

[James Stapleton-Cotton]

Thank you Mats and Steven. I'm back on track now with a different tutorial
which has lead to me to the relevant coding tools.
https://wiki.python.org/moin/BeginnersGuide/NonProgrammers
https://python.swaroopch.com

On Mon, Dec 3, 2018 at 4:04 PM Mats Wichmann  wrote:

> On 12/3/18 3:35 AM, James Stapleton-Cotton wrote:
> > Hello,
> >
> > On this page (
> >
> http://openbookproject.net/thinkcs/python/english3e/way_of_the_program.html
> )
> > - a book for learning Computer Science using Python - I am directed to (
> > http://code.google.com/p/pyscripter) in order to access the appropriate
> > program development environment, PyScripter. I have downloaded the latest
> > version of Python from Python.org and PyScripter from a site that I am
> > directed to from the original site mentioned in the beginning of this
> email
> > - (https://sourceforge.net/projects/pyscripter/files/), however I can't
> > seem to be able to run PyScripter on my Mac.
>
> it looks like from the project's own description that PyScripter is
> intended solely for the Windows environment.
>
> But there are a TON of other IDE products, free and commercial, out
> there, many of which do run on the Mac.
>
> This may be information overload, but try looking here:
>
> https://wiki.python.org/moin/IntegratedDevelopmentEnvironments
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Beginners Book, Python and PyScripter

[James Stapleton-Cotton]

Hello,

On this page (
http://openbookproject.net/thinkcs/python/english3e/way_of_the_program.html)
- a book for learning Computer Science using Python - I am directed to (
http://code.google.com/p/pyscripter) in order to access the appropriate
program development environment, PyScripter. I have downloaded the latest
version of Python from Python.org and PyScripter from a site that I am
directed to from the original site mentioned in the beginning of this email
- (https://sourceforge.net/projects/pyscripter/files/), however I can't
seem to be able to run PyScripter on my Mac.
The only file that opens is the Python Shell. I am really confused as to
whether or not PyScripter works on Mac OS or if I'm missing a step in the
installation process?

I hope I have been clear in my query. I look forward to hearing from you.

Regards,
James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Need help in learning Python

[James Gledhill via Tutor]

I know this is a python focused mail group, but you asked about Linux so I'll 
answer. :-)
I would strongly recommend that you skip Kali Linux for the next little while. 
Every tool available on Kali can be obtained on Ubuntu. Kali is not beginner 
friendly, and while the community is great, honestly it's not the most beginner 
friendly either. If you don't feel comfortable installing the tools you need 
for your course on Ubuntu, Kali is going to give you a splitting headache. 
Stick with Ubuntu, learn python, install the tools as you need them and you 
will learn so much more.

Trust me, it took me years to learn that lesson. :-|

 Original Message 
On Aug 11, 2018, 12:48 PM, Carlos Monge wrote:

> I bought two books to help me learn Python. "Python Crash Course" and
> "Python for Informatics". I have done all of the basic lessons in "Python
> Crash Course", but it has two additional sections to help instill what I
> have learned To do those sections I need to install Pygame and make sure I
> have 'pip' as well as Matplotlib. I have followed the directions in the
> book but can't get anything to download so I can use it.
> For the "Python for Informatics" book I need to have files from the
> internet load where I have Python so I can manipulate them, but I can't
> seem to get that to work.
>
> Also, I would like to install Ubuntu on a second drive with Python,
> Idle, C language and its compiler so I can also learn Linux. On the Linux
> drive, I need to install Kali Linux for a class on Cyber Security.
> I know this is a lot to ask, but I don't need it all at once. I would
> appreciate help from someone with experience in downloading all of these
> things onto a windows computer with a second drive for Ubuntu and all of
> the other software.
> Any help is appreciated, and if I find someone with the needed
> expertise, I will happily subscribe on the Python-Tutor Info Page/
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using pip

[James Reynolds]

On Thu, Jul 5, 2018 at 12:55 PM Hlavin, Matthew (GSFC-5460)[GSFC INTERNS] <
matthew.hla...@nasa.gov> wrote:

> I just downloaded Python to work on a project at work. I'm writing a
> pretty simple program for data collection for an experiment. In order to
> get the data, though I need to install PyVISA. The website for PyVISA says
> I can install the library using the line:
> $ pip install -U pyvisa
> When I type this line, I get a syntax error for using the $, and when I
> remove the $, I get a syntax error for using the word install. I even tried
> just using the word pip and an error said 'pip' is not defined. I'm not
> sure if I'm not using some syntax wrong, or if its a completely different
> issue.
>
> Thanks for any help and insight
> Matt Hlavin
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor



What version of Python did you install and what operating system are you
using?

the $ is just a command prompt and it isn't used for the above command.

The important bit from your message is the "I even tried just using the
word pip and an error said 'pip' is not defined".

This means either A.) You have pip installed, but it's not on your path or
B.) You don't have pip installed.

If you installed latest python (which is 3.7), then you can can create a
virtual env directly and just use that, which contains pip as well.

(this all assumes that python is on your path already)

python -m venv env

this will create a virtual environment called "env".

After you create your virtual environment, you activate it ".
env/bin/activate". If you are windows it would be "env\Scripts\activate"

Once activated, you can install your package like: pip install pyvisa

you may also enjoy using ipython (pip install ipython) for this kind of use
case.

James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Working example of using SSL:

[James Lundy]

Can any one get me started with SSL by providing a working example with some 
documentation?

God Bless:

James Lundy
jalu...@computer.org<mailto:jalu...@computer.org>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] calling printf() in a C language DLL using ctypes.CDLL

[James Chapman]

A long time ago when I was working with Python and DLLs I slapped together
a basic and ugly example.
You can find it here: https://github.com/James-Chapman/python-code-snippets/
tree/master/DLL_C_funcs_w_callbacks

The whole thing should load into Visual Studio. I can't guarantee that it
works in its current state though. It was created mainly as a reminder to
myself.

--
James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] List vs. Set:

[James Lundy]

To whom it may concern. This code is from Dr. John Keyser.


gooddata = []

for singleday in datalist:
if (singleday[0] == day) and (singleday[1] == month):
gooddata.append({singleday[2], singleday[3], singleday[4], 
singleday[5]})

# Perform analysis
minsofar = 120
maxsofar = -100
numgooddates = 1
sumofmin = 0
sumofmax = 0

# so singleday in datalist is a list while singleday in gooddata is a 
set?

for singleday in gooddata:

sumofmin += singleday[1]
sumofmax += singleday[2]
if singleday[1] < minsofar:
minsofar = singleday[1]
if singleday[2] > maxsofar:
maxsofar = singleday[2]

Could this be a bug in my Pycharm 3 compiler I have had mixed experiences 
running the code.

An insertion of a space after for singleday in gooddata: line 54 caused the 
program to run as desired, once, since other runs cause the error


Traceback (most recent call last):

  File "C:/Users/Dad/PycharmProjects/TopDownDesign/WeatherDataSpecialDay.py", 
line 56, in 

sumofmin += singleday[1]

TypeError: 'set' object does not support indexing

persist.

I will attach the code and test file. Please allow me to thank you in advance 
for answering this query. I am new with Python and would appreciate any advice.

God Bless:

James Lundy
jalu...@computer.org<mailto:jalu...@computer.org>
## Read in Data 
# Open file
filename = input("Enter the name of the file: ")
infile = open(filename, 'r')
#print(infile.read())






# Read in data
datalist = []

for line in infile:
#get data form line
date, h, l, r = (line.split(','))
lowtemp = int(l)
hightemp = int(h)
rainfall = float(r)
m, d, y = date.split('/')
month = int(m)
day = int(d)
year = int(y)

#put data into list
datalist.append([day, month, year, lowtemp,hightemp, rainfall])



#Close file
infile.close()

## Analyze Data 
# Get Data of interest
month = int(input("For the date you care about, enter the month:"))
day = int(input("For the date you care about, enter the day: "))
# Fomd jostproca; data fpr date
gooddata = []

for singleday in datalist:
if (singleday[0] == day) and (singleday[1] == month):
gooddata.append({singleday[2], singleday[3], singleday[4], 
singleday[5]})

# Perform analysis
minsofar = 120
maxsofar = -100
numgooddates = 1
sumofmin = 0
sumofmax = 0

# so singleday in datalist is a list while singleday in gooddata is a 
set?

for singleday in gooddata:

sumofmin += singleday[1]
sumofmax += singleday[2]
if singleday[1] < minsofar:
minsofar = singleday[1]
if singleday[2] > maxsofar:
maxsofar = singleday[2]

avglow = sumofmin / numgooddates
avghigh = sumofmax / numgooddates


## Present results 
print ("There were", numgooddates,"days")
print ("The lowest", minsofar)
print ("The highet", maxsofar)
print ("The avaerage low has been", avglow)
print ("The average high", avghigh)


## Extra code for test #
# #greeting = "Howdy!How ar you today?I'm great!"
#lines = greeting.split('')
#for line in lines:
#print(line)

1/01/2000,79,37,0
1/02/2000,79,68,0
1/03/2000,73,60,0
1/04/2000,51,26,0
1/05/2000,57,19,0
1/06/2000,59,46,0.08
1/07/2000,58,35,2.08___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When do you know you're ready to start applying for jobs?

[James Chapman]

Why has no one mentioned Github/Gitlab?

Set up a free account on either or both platforms, and start committing
your code. When applying for jobs potential employers will often want to
see what you're capable of even before inviting you for an interview, and
many will ask for a github page to see your work and whether you're
contributing to open source projects. They'll also want to see if your code
is unit tested as they'll want to employ programmers who are happy and able
to write unit tests and that understand the value of unit tests.

It also goes to show that you understand and know how to use source control
effectively as this will be a requirement for any software development
company.

https://about.gitlab.com/
https://github.com/

​Gitlab offers better features than github and it's arguable a better git
source control platform than github, that said, github has somehow become
the defacto standard for open source projects.​ At the company where I
work, we're also migrating to github enterprise.

Demand for python programmers has grown over the years, reflected in the
Tiobe index: https://www.tiobe.com/tiobe-index/

If you're able to write and understand what's happening on any of these
projects (https://github.com/trending/python) then you're ready to start
applying for jobs. Show off your skills via public git profiles and you
should have something in no time, especially if you're not fussy!
Programmers are in demand!

Hope that helps and good luck.

James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to debug a memory leak in a wsgi application?

[James Chapman]

​Ah OK, now I understand why you mentioned pymalloc to begin with.

I'm not familiar with uWSGI or cython. That said, why do you think it's
uWSGI causing a leak? It seems unlikely.
Python projects can grow in size if you're not dereferencing objects...
(see https://f0rki.at/hunting-memory-leaks-in-python.html)

If you use valgrind combined with python memory_profiler you should
hopefully be able to get an idea as to where the leak is coming from. It's
probably in your own code and leaks can be incredibly difficult to track
down. Typically while reviewing your own code you end up skipping over the
error time and time again because you become blind to your errors, so it
might help to have someone else peer review it.

These 2 links are a good starting point.
https://github.com/KratosMultiphysics/Kratos/wiki/Checking-memory-usage-with-Valgrind
https://github.com/pythonprofilers/memory_profiler

One last note, if you are doing any of your own memory allocations, then
make sure you're also freeing them:
https://cython.readthedocs.io/en/latest/src/tutorial/memory_allocation.html

But note, if you did this in cython:

cdef double* data
data =  PyMem_Malloc(100 * sizeof(double))
data =  PyMem_Malloc(100 * sizeof(double))
PyMem_Free(data)

You would (probably, you would in C/C++) end up with a leak because
you've changed the pointer. When you go to free it, only the 2nd
allocation will be freed and you'll have no way of freeing the first.

HTH
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to debug a memory leak in a wsgi application?

[James Chapman]

No, I'm saying you shouldn't need to make any kind of malloc calls
manually. Python handles memory allocation and deallocation on your behalf.
Why do you need to call pymalloc?
Are you using ctypes?
And if you are I presume this is then to make C-calls into a shared library?

James



--
James

On 13 December 2017 at 18:30, Etienne Robillard  wrote:

> Hi James,
>
> Thank for your reply. Are you suggesting that under Linux the malloc()
> glibc library call is more memory efficient than using pymalloc?
>
> Best regards,
>
> Etienne
>
> Le 2017-12-13 à 12:27, James Chapman a écrit :
>
> Why pymalloc? I presume this means you're using ctypes which means I have
> more questions.
>
> If you're allocating your own blocks of memory then you need to free them
> too. IE, does each call to pymalloc have a corresponding call to pyfree?
>
> Is the overhead of pythons built in malloc really a problem?
>
> Are you changing pointers before you've freed the corresponding block of
> memory?
>
> There are many ways to create a memory leak, all of them eliminated by
> letting python handle your memory allocations.
>
> But, back to your original question, check out "valgrind".
>
> HTH
>
> --
> James
>
> On 6 December 2017 at 16:23, Etienne Robillard  wrote:
>
>> Hi Alan,
>>
>> Thanks for the reply. I use Debian 9 with 2G of RAM and precompiled
>> Python 2.7 with pymalloc. I don't know if debugging was enabled for this
>> build and whether I should enable it to allow memory profiling with
>> guppy... My problem is that guppy won't show the heap stats for the uWSGI
>> master process. However I have partially resolved this issue by enabling
>> --reload-on-rss 200 for the uwsgi process.  Previously, the htop utility
>> indicated a 42.7% rss memory usage for 2 uWSGI processes. I have restarted
>> the worker processes with SIGINT signal. Now my uwsgi command line looks
>> like:
>>
>> % uwsgi --reload-on-rss 200 --gevent 100 --socket localhost:8000
>> --with-file /path/to/file.uwsgi --threads 2 --processes 4 --master
>> --daemonize /var/log/uwsgi.log
>>
>> My framework is Django with django-hotsauce 0.8.2 and werkzeug. The web
>> server is nginx using uWSGI with the gevent pooling handler.
>>
>> Etienne
>>
>> Le 2017-12-06 à 10:00, Alan Gauld via Tutor a écrit :
>>
>>> On 06/12/17 09:21, Etienne Robillard wrote:
>>>
>>>> Hi
>>>>
>>>> I think my wsgi application is leaking and I would like to debug it.
>>>>
>>>> What is the best way to profile memory usage in a running wsgi app?
>>>>
>>> This is probably a bit advanced for the tutor list, you might
>>> get a better response on the main Python list.
>>>
>>> But to get a sensible answer you need to provide more data:
>>> What OS and Python version?
>>> What toolset/framework are you using?
>>> What measurements lead you to suspect a memory leak?
>>>
>>>
>>>
>> --
>> Etienne Robillard
>> tkad...@yandex.com
>> https://www.isotopesoftware.ca/
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
> --
> Etienne Robillardtkadm30@yandex.comhttps://www.isotopesoftware.ca/
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to debug a memory leak in a wsgi application?

[James Chapman]

Why pymalloc? I presume this means you're using ctypes which means I have
more questions.

If you're allocating your own blocks of memory then you need to free them
too. IE, does each call to pymalloc have a corresponding call to pyfree?

Is the overhead of pythons built in malloc really a problem?

Are you changing pointers before you've freed the corresponding block of
memory?

There are many ways to create a memory leak, all of them eliminated by
letting python handle your memory allocations.

But, back to your original question, check out "valgrind".

HTH

--
James

On 6 December 2017 at 16:23, Etienne Robillard  wrote:

> Hi Alan,
>
> Thanks for the reply. I use Debian 9 with 2G of RAM and precompiled Python
> 2.7 with pymalloc. I don't know if debugging was enabled for this build and
> whether I should enable it to allow memory profiling with guppy... My
> problem is that guppy won't show the heap stats for the uWSGI master
> process. However I have partially resolved this issue by enabling
> --reload-on-rss 200 for the uwsgi process.  Previously, the htop utility
> indicated a 42.7% rss memory usage for 2 uWSGI processes. I have restarted
> the worker processes with SIGINT signal. Now my uwsgi command line looks
> like:
>
> % uwsgi --reload-on-rss 200 --gevent 100 --socket localhost:8000
> --with-file /path/to/file.uwsgi --threads 2 --processes 4 --master
> --daemonize /var/log/uwsgi.log
>
> My framework is Django with django-hotsauce 0.8.2 and werkzeug. The web
> server is nginx using uWSGI with the gevent pooling handler.
>
> Etienne
>
> Le 2017-12-06 à 10:00, Alan Gauld via Tutor a écrit :
>
>> On 06/12/17 09:21, Etienne Robillard wrote:
>>
>>> Hi
>>>
>>> I think my wsgi application is leaking and I would like to debug it.
>>>
>>> What is the best way to profile memory usage in a running wsgi app?
>>>
>> This is probably a bit advanced for the tutor list, you might
>> get a better response on the main Python list.
>>
>> But to get a sensible answer you need to provide more data:
>> What OS and Python version?
>> What toolset/framework are you using?
>> What measurements lead you to suspect a memory leak?
>>
>>
>>
> --
> Etienne Robillard
> tkad...@yandex.com
> https://www.isotopesoftware.ca/
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows Memory Basics

[James Chapman]

We're heading into advanced territory here and I might get told off but...
Consider this C++ program for a second, it has a struct with different
types of variables which sit next to each other in memory. When you print
the byte values of the struct, you can see that there is no easy way to
know which byte value belongs to which variable unless you already know the
layout.

-
#include 

typedef unsigned char  BYTE;
typedef unsigned short WORD;
typedef unsigned long  DWORD;

#pragma pack(1)
struct mem
{
char c; // 1 byte
WORD wn;// 2 byte
DWORD dwn;  // 4 byte
int i;  // 4 byte
BYTE b; // 1 byte
};

int main()
{
mem s;
s.c = 0xFF;
s.wn = 0xF;
s.dwn = 0x;
s.i = 0x;
s.b = 0xFF;

BYTE * memPointer = reinterpret_cast(&s);

for (int i = 0; i < sizeof(mem); i++)
printf("%02d [0x%08x] = %x \n", i, memPointer, *memPointer++);

return 0;
}
-

Prints

00 [0xecf0f789] = ff
01 [0xecf0f78a] = ff
02 [0xecf0f78b] = ff
03 [0xecf0f78c] = ff
04 [0xecf0f78d] = ff
05 [0xecf0f78e] = ff
06 [0xecf0f78f] = ff
07 [0xecf0f790] = ff
08 [0xecf0f791] = ff
09 [0xecf0f792] = ff
10 [0xecf0f793] = ff
11 [0xecf0f794] = ff

Packing can also come into play. If you change the packing to 2, you get
this:

00 [0x7d4ffcd9] = ff
01 [0x7d4ffcda] = cc
02 [0x7d4ffcdb] = ff
03 [0x7d4ffcdc] = ff
04 [0x7d4ffcdd] = ff
05 [0x7d4ffcde] = ff
06 [0x7d4ffcdf] = ff
07 [0x7d4ffce0] = ff
08 [0x7d4ffce1] = ff
09 [0x7d4ffce2] = ff
10 [0x7d4ffce3] = ff
11 [0x7d4ffce4] = ff
12 [0x7d4ffce5] = ff
13 [0x7d4ffce6] = cc

And if you change it to 4, you get this:

00 [0xf4f5fbf9] = ff
01 [0xf4f5fbfa] = cc
02 [0xf4f5fbfb] = ff
03 [0xf4f5fbfc] = ff
04 [0xf4f5fbfd] = ff
05 [0xf4f5fbfe] = ff
06 [0xf4f5fbff] = ff
07 [0xf4f5fc00] = ff
08 [0xf4f5fc01] = ff
09 [0xf4f5fc02] = ff
10 [0xf4f5fc03] = ff
11 [0xf4f5fc04] = ff
12 [0xf4f5fc05] = ff
13 [0xf4f5fc06] = cc
14 [0xf4f5fc07] = cc
15 [0xf4f5fc08] = cc


In other words, even if you have the source code for the program you want
to scan in memory, depending on the compiler settings the memory layout
could have changed, or rather not be what you expected due to packing and
alignment.

Probably not the answer you were hoping for but I hope this helps.

--
James




On 17 October 2017 at 01:02, Michael C 
wrote:

> Hold on, supposed by using Openprocess and VirtualQueryEx, I have the
> locations of all the memory the application is using, wouldn't this to be
> true?
>
> Say, a 8 byte data is somewhere in the region i am scanning. Ok, I know by
> scanning it like this
> for n in range(start,end,1)
>
> will read into another variable and mostly nothing, but unless a variable,
> that is, one number, can be truncated and exist in multiple locations like
> this
>
> double = 12345678
>
> 123 is at x001
> 45 is at x005
> 678 is at x010
>
> unless a number can be broken up like that, wouldn't I, while use the silly
> 'increment by one' approach,  actually luck out and get that value in it's
> actual position?
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with putting numbers from highest to lowest.

[Derry, James R]

In [2]: ?sorted
Signature: sorted(iterable, /, *, key=None, reverse=False)
Docstring:
Return a new list containing all items from the iterable in ascending order.

A custom key function can be supplied to customize the sort order, and the
reverse flag can be set to request the result in descending order.
Type:  builtin_function_or_method

In [3]: sorted([3,1,5], reverse=True)
Out[3]: [5, 3, 1]


From: Tutor [tutor-bounces+jderry=mail.utexas@python.org] on behalf of 
edmundo pierre via Tutor [tutor@python.org]
Sent: Wednesday, September 27, 2017 8:10 AM
To: Tutor Python
Subject: [Tutor] Help with putting numbers from highest to lowest.

Hello,
When I used sort() to do that, but my problem is that sort() just arrange 
numbers from small to big, not from big to small. That is the issue I am having 
now. For instance:
# The user is entering those numbers:a = 2.7b = 4.7c= 5.8d = 7.9# I will like 
the answer to be like this: 7.9  5.8  4.7 2.7
#but if I use sort(), I will have that answer, but I do not want that:2.7 4.7 
5.8 7.9
Thank you!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] overriding brackets in lvalue assignment possible?

[James Hartley]

I can successfully override __getitem__() for rvalues, but the following
example shows that more is required when used as an lvalue:

===8<-
#!/usr/bin/env python

class Foo():
def __init__(self, n):
self.d = dict.fromkeys([i for i in range(0, n)])

def __getitem__(self, i):
return self.d[i]

def main():
foo = Foo(4)
print(foo[0])
foo[0] = 2   # not as an lvalue?
print(foo[0])

if __name__ == '__main__':
main()
===8<-

Python 3.4 generates the following output when this example is executed:

None
Traceback (most recent call last):
  File "./test.py", line 17, in 
main()
  File "./test.py", line 13, in main
foo[0] = 2
TypeError: 'Foo' object does not support item assignment

I am surprised that the error states that the object itself cannot accept
assignment.  From the C++ perspective, the underlying dictionary should be
exposed.  Does Python overloading allow use of bracket overriding in
lvalues?

Thanks.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] enforcing specific types in Python 3.5?

[James Hartley]

I have implemented the equivalent of "insert if unique" in Python &
SQLAlchemy to help with data normalization.  However to help minimize the
number of preliminary SELECT statements needed, it helps to check types
through calls to isinstance() before getting to the salient code.
Unfortunately, the code begins to be cluttered with type-checking minutiae.

While researching this problem, I have found potential solutions like the
following:

http://stackoverflow.com/questions/9305751/force-python-class-member-variable-to-be-specific-type

...but given that Python 3.5 has syntactically understands gradual typing,
I have wondered whether addition of this feature offers anything over use
of property() as described above.  Toy examples have not revealed anything
useful on this front:

$ cat example.py
#!/usr/bin/env python

class Foobar():
def __init__(self, value):
self.value = value

def f(s: str) -> int:
print(s)
return 3.14

def main():
i = f('foobar')
print(type(i))
print('i = "{}"'.format(i))
i = f(Foobar(3))
print(type(i))
print('i = "{}"'.format(i))

if __name__ == '__main__':
main()
$ python example.py
foobar

i = "3.14"
<__main__.Foobar object at 0x85b8aaac>

i = "3.14"
$

I understand that gradual typing may be useful with static analysis, but I
don't see that any type enforcement occurs by default at runtime.  Am I
missing something here?   Is there a better solution for type enforcement?

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Code for python game

[Derry, James R]

hi, tracey,

are you allowed to use python sets? if so, you should take a look at them and 
their methods:
https://docs.python.org/3.5/tutorial/datastructures.html?highlight=data%20structures#sets

best,
james

From: Tutor [tutor-bounces+jderry=mail.utexas@python.org] on behalf of 
tracey jones-Francis [drtraceyjo...@hotmail.com]
Sent: Wednesday, October 12, 2016 12:40 PM
To: tutor@python.org
Subject: [Tutor] Code for python game

Im in the middle of designing a game in python, im quite new to using it and am 
struggling with a certain bit of code. I want to have a function that will 
ignore certain words that i have specified in a dictionary. I have tried so 
many different types of code but can't seem to return just the keywords i want.


the dictionary is called skip_words and has about 20 different strings in.


so far i came up with:

def filter_words(words, skip_words):

   word = words.strip(skip_words)


   return word


The function should read an input such as help me please and filter out the 
words "please" and "me" as they are in the dictionary and just return the word 
help as the input.


>>>filter_words(["help", "me", "please"], ["me", "please"])

['help']


That doesn't work but i just assume it is a very simple bit of code.

any suggestions of a way that works?


Thanks:) T
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modularity

[James Chapman]

I should have re-read that last reply before hitting send. Apologies
for the poor sentence construction!

Something I forgot to highlight before which might be related to your
initial question.

If you have a file called sound.py which contained a class called
WavFile, if you imported just sound like this:

import sound

Then your class constructor would be called like this:

wavFile = sound.WavFile()


Importing the module doesn't import the class, for that you'd do

from sound import WavFile

wavFile = WavFile()
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Procedure to install dlib on windows?

[James Chapman]

>From one of the Python examples:

# COMPILING/INSTALLING THE DLIB PYTHON INTERFACE
#   You can install dlib using the command:
#   pip install dlib
#
#   Alternatively, if you want to compile dlib yourself then go into the dlib
#   root folder and run:
#   python setup.py install
#   or
#   python setup.py install --yes USE_AVX_INSTRUCTIONS

It looks like the pip installer will install a pre-compiled lib with
python API bindings for you.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modularity

[James Chapman]

May I suggest: https://docs.python.org/2/tutorial/modules.html

In particular:
* https://docs.python.org/2/tutorial/modules.html#the-module-search-path
* https://docs.python.org/2/tutorial/modules.html#packages


Now the next bit of advice is likely to be controversial but I have
good reasons for it.

I like to import the top level module and use the full namespace in
the code, that way, when I come back to it in 3 years time I know
where each function call is coming from.

For example, lets say I had the following package (stolen from the
docs page linked above):


sound/  Top-level package
  __init__.py   Initialize the sound package
  formats/  Subpackage for file format conversions
  __init__.py
  wavread.py
  wavwrite.py
  aiffread.py
  aiffwrite.py
  auread.py
  auwrite.py
  ...
  effects/  Subpackage for sound effects
  __init__.py
  echo.py
  surround.py
  reverse.py
  ...
  filters/  Subpackage for filters
  __init__.py
  equalizer.py
  vocoder.py
  karaoke.py


I myself would import as follows

import sound

Then in my code, the calls would look like:

wave = sound.formats.waveread(someFile)
aiffFile = sound.formats.aiffwrite(wave)
auFile = sound.formats.auwrite(wave)


If I did:

from sound import formats.*

Then the code would be

wave = waveread(someFile)
aiffFile = aiffwrite(wave)
auFile = auwrite(wave)


The problem with the latter is, which module supplies waveread?
I have roughly 20 import statements where I'm importing something.*
which one of those modules supplies the function waveread?


A way around the above would be

import sound.formats.waveread
import sound.formats.aiffwrite
import sound.formats.auwrite

Code would then be

wave = waveread(someFile)
aiffFile = aiffwrite(wave)
auFile = auwrite(wave)



But what if we have various modules that implement a waveread function?

Then we'd have to start using

import sound.formats.waveread as sfwaveread
import some.other.waveread


Code is read more than it is written, so don't be lazy! Use the
namespaces in your code. You make it clear what you're doing it you
avoid clashing.


Finally, if you absolutely must be lazy, then import like this:

from sound import formats as sf

wave = sf.waveread(someFile)
aiffFile = sf.aiffwrite(wave)
auFile = sf.auwrite(wave)


There is nothing I hate more than being asked to change or fix someone
elses code when that programmer is lazy and feels he/she can produce a
solution quicker by reducing the amount of typing he/she has to do.
Well, that's a lie, there are things I hate more but this come close!


James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question about the memory manager

[James Chapman]

If you read the comment that goes with the code snippet pasted in the
original email it makes far more sense as the author is talking
specifically about out of memory errors...


"You already got excellent answers, I just wanted to add one more tip
that's served me well over the years in a variety of language for the
specific problem "how to cleanly diagnose, log, etc, out of memory
errors?". Problem is, if your code gets control before enough objects
have been destroyed and their memory recycled, memory might be too
tight to do propert logging, gui work, etc, etc -- how do we ensure
this doesn't happen?

Answer: build an emergency stash so you know you can spend it in such
emergencies:

rainydayfund = [[] for x in xrange(16*1024)]  # or however much you need

def handle_exception(e):
  global rainydayfund
  del rainydayfund
  ... etc, etc ...

" - Alex Martelli


--
James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] reading an input stream

[James Chapman]

Hi Richard

There are a number of considerations you need to take into account here.

Raw sockets is almost never the right solution, while a basic socket to
socket connection is easy enough to program, handling failure and
concurrency can very quickly make the solution a lot more complex than it
needs to be, so perhaps you could supply more information? (I realise I'm
venturing outside the realm of learning python, but I'm a pedant for doing
things right).

You said you need to read XML in from a socket connection. You've not
mentioned what's generating the data? Is that data sent over HTTP in which
case is this part of a SOAP or REST API? Is the data being generated by
something you've written or a 3rd party software package? Is REST an
option? Is there a reason to serialise to XML? (If I was performing the
serialisation I would go with JSON if being human readable was a
requirement. )

If the method of receiving that data is optional, have you considered using
something like AMQP (RabbitMQ) which would eliminate your need to support
concurrency? It would also handle failure well.

James


--
James

On 29 December 2015 at 20:14, richard kappler  wrote:

> Sorry it took so long to respond, just getting back from the holidays. You
> all have given me much to think about. I've read all the messages through
> once, now I need to go trough them again and try to apply the ideas. I'll
> be posting other questions as I run into problems. BTW, Danny, best
> explanation of generators I've heard, well done and thank you.
>
> regards, Richard
>
> On Thu, Dec 24, 2015 at 4:54 PM, Danny Yoo  wrote:
>
> > > I think what I need to do would be analogous to (pardon if I'm using
> the
> > > wrong terminology, at this poing in the discussion I am officially out
> of
> > > my depth) sending the input stream to a buffer(s) until  the ETX for
> that
> > > message comes in, shoot the buffer contents to the parser while
> accepting
> > > the next STX + message fragment into the buffer, or something
> analogous.
> >
> > Yes, I agree.  It sounds like you have one process read the socket and
> > collect chunks of bytes delimited by the STX markers.  It can then
> > send those chunks to the XML parser.
> >
> >
> > We can imagine one process that reads the socket and spits out a list
> > of byte chunks:
> >
> > chunks = readDelimitedChunks(socket)
> >
> > and another process that parses those chunks and does something with
> them:
> >
> > for chunk in chunks:
> > 
> >
> >
> > It would be nice if we could organize the program like this.  But one
> > problem is that chunks might not be finite!  The socket might keep on
> > returning bytes.  If it keeps returning bytes, we can't possibly
> > return a finite list of the chunked bytes.
> >
> >
> > What we really want is something like:
> >
> > chunkStream = readDelimitedChunks(socket)
> > for chunk in chunkStream:
> > 
> >
> > where chunkStream is itself like a socket: it should be something that
> > we can repeatedly read from as if it were potentially infinite.
> >
> >
> > We can actually do this, and it isn't too bad.  There's a mechanism in
> > Python called a generator that allows us to write function-like things
> > that consume streams of input and produce streams of output.  Here's a
> > brief introduction to them.
> >
> > For example, here's a generator that knows how to produce an infinite
> > stream of numbers:
> >
> > ##
> > def nums():
> > n = 0
> > while True:
> > yield n
> > n += 1
> > ##
> >
> > What distinguishes a generator from a regular function?  The use of
> > "yield".  A "yield" is like a return, but rather than completely
> > escape out of the function with the return value, this generator will
> > remember what it was doing  at that time.  Why?  Because it can
> > *resume* itself when we try to get another value out of the generator.
> >
> > Let's try it out:
> >
> > #
> >
> > >>> numStream = nums()
> > >>> numStream.next()
> > 0
> > >>> numStream.next()
> > 1
> > >>> numStream.next()
> > 2
> > >>> numStream.next()
> > 3
> > >>> numStream.next()
> > 4
> > #
> >
> > Every next() we call on a generator will restart it from where it left
> > off, until it reaches 

[Tutor] Help with returning a list object from a C extension.

[James Oren]

Hi all, this is my first time using the mailing list.

I'm trying to learn how to use C to extend my code, and have already worked
out how to use ctypes. I'm now trying to learn the full C extension module
approach with Python.h and have worked through the Python doc and a couple
other examples. They all use examples where the C function expects a
primitive, so now I'd like to learn how to use a Python list object. I
guess that a C-based numpy array may be easier to write for, but just for
now I'd like to stick to lists.

So far I've worked out to use the parse tuple command with the "O" argument
and I think thanks to an archived post on this mailing list I've worked out
how to get the Python list data into a C array and pass it to the function
to modify, now I just need to return a list object of the updated C array
data.

I'd like it to be that calling this function from Python has the same
effect as modifying the list in-place, so I guess I need to either empty
the original and repopulate it with the c array data, or build a new list
object with the data and reassign it to the original list name  Which
seems like it'd be troublesome? Unfortunately I've hit a brick wall and
can't find any examples :-(

Any help would be greatly appreciated!

Jim
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] relative imports within a package?

[James Hartley]

The Python 3 tutorial discusses relative imports at:

https://docs.python.org/3/tutorial/modules.html#intra-package-references

I have the following directory structure for a package in development:

+ outer_package/
 + __init__.py
 + inner_package
 | + __init__.py
 | + myclass.py
 + collateral_directory
   + arbitrary_tool.py

arbitrary_tool.py needs to instantiate the class MyClass found in
myclass.py.  I have made MyClass accessible by adding the following to
inner_package/__init__.py:

from myclass import MyClass

Within arbitrary_tool.py, I have attempted the following:

from ..inner_package import MyClass

...but executing with Python 3.4 yields the following stack trace:

Traceback (most recent call last):
  File "./arbitrary_tool.py", line 5, in 
from ..inner_package import MyClass
SystemError: Parent module '' not loaded, cannot perform relative import

Suggestions on how to solve this would be most certainly welcomed!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] accessing modules found throughout a package?

[James Hartley]

In my current project, I am developing a package.  It makes sense to embed
tests throughout the package's directory structure as they should be part
of the package & its distribution.  It may raise eyebrows that I have tests
sprinkled through various directories, but there are reasons for keeping
some tests in the same proximity as the modules needed to import data into
a database.  There are several directories importing data according to
different constraints.

The problem/annoyance I am facing is that tests need to access modules in
other directories, I have to play games at the beginning of each test file
which looks at os.path.realpath(__file__) to ascertain where the root of
the package can be found before calling os.path.append().  Since the
package is still under development, its structure is still in flux.  As a
result, any structural change requires that I re-adjust each test file
ensuring that the paths are still correct.

Here is the package's general structure:

+/package_directory/
   |
  +/data/
   + base_classes_used_by_some_tests.py
|
   +/import_1/
|+ test_code_requiring_base_classes_above.py
|
   +/import_2/
| + different_test_code_requiring_base_classes_above.py
|
   +/tests/
  |
 + more_test_code_requiring_base_classes_above.py

What is a better solution to minimize any tweaking forced upon the test
code?  It seems that formally installing the package will only remedy some
of the problems I am currently experiencing.

Any suggestions you may have will be very welcomed.

Jim
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Topic focus of ‘python-tutor’ (was: mySQL and Python)

[James Chapman]

Long-ish reply, but please bear with me.

To quote the list description "This list is for folks who want to ask
questions regarding how to learn computer programming with the Python
language and its standard library."

While MySQL modules are not part of the standard library, consider the
following scenario.

A person, let's call him Xerxes, is interesting in learning to program in
Python. So he does a whole lot of reading, asks a load of questions on this
list, he learns the basics. He now thinks to himself, wow, it would be
really cool if I could actually do something really useful with my new
skill set, maybe he has a job where building an interface to a database
would save him and his company a lot of effort, but as MySQL is not part of
the standard library he's not really sure how to go about it. Since people
on the tutor list have always been willing to help and offer assistance, he
thinks, I'll ask there. He asks his question and almost immediately gets
told that what he's asking is off topic for the list. Let's jump back to
the list description for a minute there. "This list is for folks who want
to *ask questions regarding how to learn computer programming with the
Python language* and its standard library." Is the installation and usage
of 3rd party modules not related to learning to program with python?

By responding in that manner, you're discouraging the asking of questions.
No one should *_EVER_ *be discouraged to ask a question they do not know
the answer to. That response that I referred to in my initial response is
very common on this list and I think we (collectively) could and should
come up with a better way of pointing people in the direction of
satisfactory resolution. I also don't think we should immediately point
people else where when the question is outside our comfort zone. OO,
inheritance and multi-byte string processing, to name a few, are part of
learning to program but are easily more advanced topics than which MySQL
module to use. Creating C data structures in python, while part of the
standard library, is an example of a topic that is probably too advanced
for this list, and even then, I managed to find someone on this list able
to help with that topic.

If we discourage people from asking more interesting questions then I
suspect that many subscribers will stop subscribing and that knowledge will
be lost.

I subscribe to quite a few programming related mailing lists and let me
tell you, this one is _BY_FAR_ the most willing to help. That question
would very likely have been ignored on other "more advanced" lists.

Finally, I realise I've offered some criticism but not a suggestion for an
alternative, so here is an example of a response to a question that would
be considered "outside the scope" of this list:


Hi Xerxes

Great question. I know there are many options when it comes to MySQL but
I've not had any experience with any. Someone else on this list might be
able to provide a satisfactory answer, but, as the main focus of this list
is learning to program in python using the standard library you might not
get an answer, and if you do, the answer might not be very knowledgeable.
You could try asking your question on forum X or mailing list Y as these
types of topics are often covered there.

James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] mySQL and Python

[James Chapman]

One of my pet hates about this list... "This is a tutor list, your question
is out of scope". Sure there might be better places to seek answers, and
sure maybe the first responder doesn't know the answer, but that's not a
reason to respond with that phrase. This list is a called python tutor, not
python beginner, even if the large majority of the questions are beginner
questions. The fact that people can ask any python related question is one
of the things I like about it and wish that other languages had similar
lists.

Back to answer the original question...

I recommend using the official MySQL connector because it's supported by
MySQL and it's continuously developed, which means it won't stop working
when you change Python versions, or MySQL versions, and it's documented.
I've tried some other MySQL libs in the past that worked OK but were a
nightmare when it came to supporting them due to changes in the environment.

Download the connector from
https://dev.mysql.com/downloads/connector/python/

The latest version should work just fine on Python 3.4

Documentation on how to use it is also available on the MySQL website:
https://dev.mysql.com/doc/connector-python/en/connector-python-examples.html

Hope this helps.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Assistance with UnicodeDecodeError

[James Chapman]

Actually, it's more likely that the char you are grabbing is UTF-16 not
UTF-8 which is moving into the double byte...
* An assumption based on the following output:

>>> u = u'\u2014'
>>> s = u.encode("utf-16")
>>> print(s)
 ■¶
>>> s = u.encode("utf-32")
>>> print(s)
 ■  ¶
>>> s = u.encode("utf-16LE")
>>> print(s)
¶
>>> s = u.encode("utf-16BE")
>>> print(s)
 ¶

See https://en.wikipedia.org/wiki/Character_encoding to help with the
understanding of character encoding, code pages and why they are important.





James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Assistance with UnicodeDecodeError

[James Chapman]

>
> I am trying to scrap text from a website using Python 2.7 in windows 8 and
> i am getting this error *"**UnicodeDecodeError: 'charmap codec can't encode
> character u'\u2014 in position 11231 character maps to "*
>
>
For starters, move away from Python 2 unless you have a good reason to use
it. Unicode is built into Python 3 whereas it's an after thought in Python
2.

What's happening is that python doesn't understand the character set in use
and it's throwing the exception. You need to tell python what encoding to
use: (not all website are "utf-8")


Code example (using python 2.7):

>>> u = u'\u2014'
>>> print(u)
Traceback (most recent call last):
  File "", line 1, in 
  File "c:\Python27\lib\encodings\cp850.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u2014' in
position 0: character maps to 
>>> s = u.encode("utf-8")
>>> print(s)
ÔÇö



I also strongly suggest you read:
https://docs.python.org/2/howto/unicode.html

There is much cursing to come. Unicode and especially multi-byte character
string processing is a nightmare!
Good luck ;-)

James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python 3.4.1 ImportError Linux

[James Chapman]

> Further to my last email, here's some reading regarding Python Paths
>
>
http://www.stereoplex.com/blog/understanding-imports-and-pythonpath
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python 3.4.1 ImportError Linux

[James Chapman]

cd ..

Terminal (~/lorem): python3 app/main.py

import statement is relative to pwd.


--
James

On 16 December 2014 at 14:18, Juan Christian 
wrote:
>
> Python 3.4.1
> Fedora 21 Server
>
> My paths:
> ~/lorem
> ~/lorem/app
> ~/lorem/core
>
> I want to execute: ~/lorem/app/main.py
>
> Terminal (~/lorem/app): python3 main.py
>
> Traceback (most recent call last):
> File "app/main.py", line 5, in 
> from core.backpack import BackpackThread
> ImportError: No module named 'core'
>
>
> Why am I getting this?
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] While Loop Help

[James Chapman]

While Alan has given you a far better solution, I feel someone should
mention the break statement as you will likely come across it a lot, and it
is quite an important flow control statement.

You could add a break statement to the else which would break out of the
while loop.
https://docs.python.org/3.4/reference/simple_stmts.html#break

Refactored to include a break

--

...

the_number = random.randint(1, 100)
win = false
tries = 0
guess = int(input("Take a guess: "))

while tries < 10:
guess = int(input("Take a guess: "))
tries += 1
if guess > the_number:
print("Lower...")
elif guess < the_number:
print("Higher...")
else:
win = true
break

if win:
print("You win")
else:
print("You fail! The number was {0}".format(the_number))

input("\n\nPress the enter key to exit.")
--
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Does the user need to install Python, when we deploy our c++ products using python?

[James Chapman]

On 2 December 2014 at 20:28, gordon zhang  wrote:

>
>
> I downloaded python 3.4.2 for c++ and create a vc++ project using python,
> but I have no idea what python dlls and other stuff needed to deploy the
> products.
>
> I know if we put Python34.dll and Python.dll in the folder of executable,
> it is not enough. What else do we need to put in the folder of
> executable?(the application does not run)
>
> If anyone knows please let me know.
>
> Thanks, Gordon
>

Probably a little advanced for this list but the list is here to learn
right? So my own 2 pence worth...

Are you referring to Python Tools for Visual Studio?
http://pytools.codeplex.com/

This allows you to build python projects in VS and then debug them using
the VS debugger, but you will need the debug files (.pdb) available on this
page: https://www.python.org/downloads/windows/ - This however is more for
extending python via C/C++ than building python apps or using python to
deploy apps.

There are a number of tools out there, some already mentioned for bundling
python apps into exe's which then require the python dlls to be packaged in
an installer to be deployed with your newly built exe. I have done this in
the past and experience has taught me one thing. Don't do it! You end up
making your app way too complex, and it becomes very difficult to support.
Also, when you want to update the python environment you run into problems.
>From someone who has been there, don't do it. Even if you have CI
infrastructure in place to make things easily repeatable, it still becomes
a nightmare.

Rather find an automated way of deploying python independently and then use
an installer to install just your python code. That way all you support is
your code, you don't end up supporting 3rd party wrappers that don't work
properly.

nullsoft scriptable installer is easy to use, has good documentation and
examples, and doesn't have a massively steep learning curve.
http://nsis.sourceforge.net/Main_Page
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Does the user need to install Python, when we deploy our c++ products using python?

[James Chapman]

Actually, after re-reading your original message I think I misunderstood
and went off on a tangent.

If you are embedding python into a C++ app, then you will need the dlls and
all of the compiled python code. *.pyc. (Well most, not all). The exact
location of these files in the final build I'm unsure of.



--
James

On 11 December 2014 at 11:39, James Chapman  wrote:

>
> On 2 December 2014 at 20:28, gordon zhang 
> wrote:
>
>>
>>
>> I downloaded python 3.4.2 for c++ and create a vc++ project using python,
>> but I have no idea what python dlls and other stuff needed to deploy the
>> products.
>>
>> I know if we put Python34.dll and Python.dll in the folder of executable,
>> it is not enough. What else do we need to put in the folder of
>> executable?(the application does not run)
>>
>> If anyone knows please let me know.
>>
>> Thanks, Gordon
>>
>
> Probably a little advanced for this list but the list is here to learn
> right? So my own 2 pence worth...
>
> Are you referring to Python Tools for Visual Studio?
> http://pytools.codeplex.com/
>
> This allows you to build python projects in VS and then debug them using
> the VS debugger, but you will need the debug files (.pdb) available on this
> page: https://www.python.org/downloads/windows/ - This however is more
> for extending python via C/C++ than building python apps or using python to
> deploy apps.
>
> There are a number of tools out there, some already mentioned for bundling
> python apps into exe's which then require the python dlls to be packaged in
> an installer to be deployed with your newly built exe. I have done this in
> the past and experience has taught me one thing. Don't do it! You end up
> making your app way too complex, and it becomes very difficult to support.
> Also, when you want to update the python environment you run into problems.
> From someone who has been there, don't do it. Even if you have CI
> infrastructure in place to make things easily repeatable, it still becomes
> a nightmare.
>
> Rather find an automated way of deploying python independently and then
> use an installer to install just your python code. That way all you support
> is your code, you don't end up supporting 3rd party wrappers that don't
> work properly.
>
> nullsoft scriptable installer is easy to use, has good documentation and
> examples, and doesn't have a massively steep learning curve.
> http://nsis.sourceforge.net/Main_Page
>
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] break and exit

[James Rieve]

I accidently used 'exit' in a loop where I meant to use 'break' and, in that
case, the program seemed to work as expected but in some cases 'exit' seems
to behave differently from 'break'. For example, in this code snippet using
'exit' or 'break' produces the same result:

for i in range(10):
if i > 3:
exit
else:
print(i)
print('out of loop')

But in this case they behave differently:

for i in range(10):
if i > 3:
break   # try using exit here.
else:
print(i)
else:
print('for loop else statement')

print('out of loop')

Does anyone have any pointers to descriptions of 'exit', what it is, what it
means, how It's used, etc.?


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] While Loop and Threads

[James Chapman]

Apologies, I didn't answer your question.

while time_now  < time_finish: # counter and count_until defined
somewhere above and
if g_threadStop.is_set() == False:
# return something

Thread methods are typically void, meaning they return nothing. At least
this is the case in other languages, so I would expect the return to fail.
Also, what would you be returning it to?
Perhaps just a return without a value to indicate exit. (Probably some
reading required here)

Exceptions should be raised when something occurs that you were not
expecting and not used as an exit mechanism for normal operation.

Where to place the timer is up to you. Another thread is an option, but is
it necessary? Overhead needs to be considered.


--
James


On 15 July 2014 14:14, James Chapman  wrote:

> So if I understand this correctly, you want to start a thread and then
> stop it after a certain time period?
>
> Here's an adapted example that includes a timer. (I believe in learning by
> example where possible)
> With that said, if it were my code going into production I'd move the
> timer logic out of the thread and only signal the thread when it should
> exit. Keep the logic within the thread simple and try and make each
> method/function do one thing only. It makes for testable, reusable code.
>
> In my new example
> * The first thread should exit because the time limit is met.
> * The second thread should exit because the Event is set.
>
> Hope this helps...
>
> James
>
>
>
> import time
> import threading
>
> class ThreadExample(object):
>
> def __init__(self):
> self.thread1Stop = threading.Event()
> self.thread2Stop = threading.Event()
> self.timeFinish = time.time() + 12
>
> def threadWorkerOne(self, _arg1, _arg2):
> print("THREAD1 - started with args: %s, %s" % (_arg1, _arg2))
> while(not self.thread1Stop.is_set()):
> while(time.time() < self.timeFinish):
> print("THREAD1 - Running")
> if (self.thread1Stop.is_set()):
> break # Break out of the time while loop
> time.sleep(1)
> if time.time() > self.timeFinish:
> print("THREAD1 - Time limit exceeded")
> self.thread1Stop.set()
> print("THREAD1 - Exiting because thread1Stop is TRUE")
>
> def threadWorkerTwo(self, _arg1, _arg2):
> print("THREAD2 - started with args: %s, %s" % (_arg1, _arg2))
> while(not self.thread2Stop.is_set()):
> while(time.time() < self.timeFinish):
> print("THREAD2 - Running")
> if (self.thread2Stop.is_set()):
> break # Break out of the time while loop
> time.sleep(1)
> if time.time() > self.timeFinish:
> print("THREAD2 - Time limit exceeded")
> self.thread2Stop.set()
> print("THREAD2 - Exiting because thread2Stop is TRUE")
>
> def main(self):
> t1 = threading.Thread(target=self.threadWorkerOne, args = ("arg1",
> "arg2"))
> t1.start()
> t2 = threading.Thread(target=self.threadWorkerTwo, args = ("arg1",
> "arg2"))
> t2.start()
> print("MAIN - sleep(5)")
> time.sleep(5)
> print("MAIN - setting thread1Stop")
> self.thread1Stop.set()
> print("MAIN - sleep(10)")
> time.sleep(10)
> print("MAIN - exiting...")
>
> if __name__ == '__main__':
> runExample = ThreadExample()
> runExample.main()
>
>
>
>
>
>
>
> --
> James
>
>
> On 15 July 2014 11:09, Oğuzhan Öğreden  wrote:
>
>> Thanks!
>>
>> I'll have a side question. If I implement this idea to my case,
>> threadWorker() would look like this:
>>
>>
>> def threadWorker(_arg1, _arg2):
>> print("Starting worker thread with args: %s, %s" % (_arg1, _arg2))
>> while g_threadStop.is_set() == False:
>> ## here comes a for loop:
>> while time_now < time_finish: # time_finish and counter
>> defined somewhere above and
>> print("Thread running.")  # and expression of the
>> condition is intentionally (and necessarily, I believe)
>> counter += 1 # left out of for
>> expression.
>> print(counter)
>> time.sleep(1)
>> print("Thread exiting...")
>>

Re: [Tutor] While Loop and Threads

[James Chapman]

So if I understand this correctly, you want to start a thread and then stop
it after a certain time period?

Here's an adapted example that includes a timer. (I believe in learning by
example where possible)
With that said, if it were my code going into production I'd move the timer
logic out of the thread and only signal the thread when it should exit.
Keep the logic within the thread simple and try and make each
method/function do one thing only. It makes for testable, reusable code.

In my new example
* The first thread should exit because the time limit is met.
* The second thread should exit because the Event is set.

Hope this helps...

James



import time
import threading

class ThreadExample(object):

def __init__(self):
self.thread1Stop = threading.Event()
self.thread2Stop = threading.Event()
self.timeFinish = time.time() + 12

def threadWorkerOne(self, _arg1, _arg2):
print("THREAD1 - started with args: %s, %s" % (_arg1, _arg2))
while(not self.thread1Stop.is_set()):
while(time.time() < self.timeFinish):
print("THREAD1 - Running")
if (self.thread1Stop.is_set()):
break # Break out of the time while loop
time.sleep(1)
if time.time() > self.timeFinish:
print("THREAD1 - Time limit exceeded")
self.thread1Stop.set()
print("THREAD1 - Exiting because thread1Stop is TRUE")

def threadWorkerTwo(self, _arg1, _arg2):
print("THREAD2 - started with args: %s, %s" % (_arg1, _arg2))
while(not self.thread2Stop.is_set()):
while(time.time() < self.timeFinish):
print("THREAD2 - Running")
if (self.thread2Stop.is_set()):
break # Break out of the time while loop
time.sleep(1)
if time.time() > self.timeFinish:
print("THREAD2 - Time limit exceeded")
self.thread2Stop.set()
print("THREAD2 - Exiting because thread2Stop is TRUE")

def main(self):
t1 = threading.Thread(target=self.threadWorkerOne, args = ("arg1",
"arg2"))
t1.start()
t2 = threading.Thread(target=self.threadWorkerTwo, args = ("arg1",
"arg2"))
t2.start()
print("MAIN - sleep(5)")
time.sleep(5)
print("MAIN - setting thread1Stop")
self.thread1Stop.set()
print("MAIN - sleep(10)")
time.sleep(10)
print("MAIN - exiting...")

if __name__ == '__main__':
runExample = ThreadExample()
runExample.main()







--
James


On 15 July 2014 11:09, Oğuzhan Öğreden  wrote:

> Thanks!
>
> I'll have a side question. If I implement this idea to my case,
> threadWorker() would look like this:
>
>
> def threadWorker(_arg1, _arg2):
> print("Starting worker thread with args: %s, %s" % (_arg1, _arg2))
> while g_threadStop.is_set() == False:
> ## here comes a for loop:
> while time_now < time_finish: # time_finish and counter
> defined somewhere above and
> print("Thread running.")  # and expression of the
> condition is intentionally (and necessarily, I believe)
> counter += 1 # left out of for
> expression.
> print(counter)
> time.sleep(1)
> print("Thread exiting...")
>
> Of course in this case value of g_threadStop.is_set() is not checked until
> for loop finishes its iteration. Would the suggestion below be a good way
> to handle the event signal?
>
>
> def threadWorker(_arg1, _arg2):
> print("Starting worker thread with args: %s, %s" % (_arg1, _arg2))
> ## here comes a for loop:
> while time_now  < time_finish: # counter and count_until defined
> somewhere above and
> if g_threadStop.is_set() == False:
> # return something or raise an exception to signal
> iteration was interrupted
> print("Thread running.")   # and expression of the condition
> is intentionally (and necessarily, I believe)
> counter += 1  # left out of for expression.
> time.sleep(1)
> print("Iteration done.")
>
> This comes with a problem, due to time.sleep(1), signal is handled with a
> delay. I can think of a work around through GTK:
>
>1. changing (1) to something like while time_now < time_finish -
>actually that is the way it is in original code.
>2. dealing with the counter in the GTK main loop since it's main use
>is for GTK display of the timer.
&g

Re: [Tutor] While Loop and Threads

[James Chapman]

OK, so I mocked up an example now...



import time
import threading

g_threadStop = threading.Event()

def threadWorker(_arg1, _arg2):
print("Starting worker thread with args: %s, %s" % (_arg1, _arg2))
while(not g_threadStop.is_set()):
print("Thread running.")
time.sleep(1)
print("Thread exiting...")

def main():
t = threading.Thread(target=threadWorker, args = ("arg1", "arg2"))
t.start()
print("Main thread sleeping for 10 seconds...")
time.sleep(5)
print("Main thread setting g_threadStop")
g_threadStop.set()
time.sleep(3)
print("Main thread exiting...")

if __name__ == '__main__':
main()





--
James


On 14 July 2014 19:03, James Chapman  wrote:

> Multi-threading takes practice!
>
> Are you using an event object to signal the thread should exit? I'm
> guessing you're just using a bool which is why it does not work.
>
> See: https://docs.python.org/3.4/library/threading.html#event-objects
>
> I'm very short on time and the moment and therefore can't mock up a
> working example. If I have time later/tomorrow and you haven't solved it or
> no one else has commented I'll try and put something together.
>
> IMO, move away from GTK until you get threading working as expected, then
> add the additional layer. Solve one problem at a time.
>
> James
>
> --
> James
>
>
> On 13 July 2014 12:23, Oğuzhan Öğreden  wrote:
>
>> Hi,
>>
>> I've been practicing with multithreading and gtk for a while and recently
>> have observed something I can't quite grasp.
>>
>> This is basically a timer with a settings window and a countdown window
>> which is produced after setting_window passes necessary arguments to thread.
>>
>> I have a while loop which counts time, inside the thread, while variable
>> "runner" is True, which looks like this:
>>
>> def run(self):
>> cycle = ['work', 'break', 'long_break']
>> cycle = cycle[:2]*self.workn+cycle[3]
>> self.runner = True
>> sec_count = timedelta(seconds=1)
>>
>> while self.runner == True:
>> for i in cycle:
>> self.setState(i)
>> print('Start:', i, datetime.now())
>>
>> while self.end_time > datetime.now():
>> time.sleep(1)
>> self.duration -= sec_count
>>
>> self.count[self.state] += 1
>>
>> And I want countdown to stop when countdown_window receives delete event,
>> thus I have a countdown_window.stop() function which is defined as follows:
>>
>> def stop(self, *args):
>> pom.count[pom.state] -= 1
>> pom.runner = False # refers to the runner of the thread.
>> print(pom.runner) # prints False,
>> Gtk.main_quit()
>>
>> What I expect to happen is that while loop to break, but it does not.
>>
>> Any suggestions?
>>
>> Best,
>> Oğuzhan
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] While Loop and Threads

[James Chapman]

Multi-threading takes practice!

Are you using an event object to signal the thread should exit? I'm
guessing you're just using a bool which is why it does not work.

See: https://docs.python.org/3.4/library/threading.html#event-objects

I'm very short on time and the moment and therefore can't mock up a working
example. If I have time later/tomorrow and you haven't solved it or no one
else has commented I'll try and put something together.

IMO, move away from GTK until you get threading working as expected, then
add the additional layer. Solve one problem at a time.

James

--
James


On 13 July 2014 12:23, Oğuzhan Öğreden  wrote:

> Hi,
>
> I've been practicing with multithreading and gtk for a while and recently
> have observed something I can't quite grasp.
>
> This is basically a timer with a settings window and a countdown window
> which is produced after setting_window passes necessary arguments to thread.
>
> I have a while loop which counts time, inside the thread, while variable
> "runner" is True, which looks like this:
>
> def run(self):
> cycle = ['work', 'break', 'long_break']
> cycle = cycle[:2]*self.workn+cycle[3]
> self.runner = True
> sec_count = timedelta(seconds=1)
>
> while self.runner == True:
> for i in cycle:
> self.setState(i)
> print('Start:', i, datetime.now())
>
> while self.end_time > datetime.now():
> time.sleep(1)
> self.duration -= sec_count
>
> self.count[self.state] += 1
>
> And I want countdown to stop when countdown_window receives delete event,
> thus I have a countdown_window.stop() function which is defined as follows:
>
> def stop(self, *args):
> pom.count[pom.state] -= 1
> pom.runner = False # refers to the runner of the thread.
> print(pom.runner) # prints False,
> Gtk.main_quit()
>
> What I expect to happen is that while loop to break, but it does not.
>
> Any suggestions?
>
> Best,
> Oğuzhan
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How can I let the Python Console display more decimal precision?

[James Chapman]

You could just initialise your variables with float()

>>> float(26)/float(12)
2.1665

>>> varA = float(26)
>>> varB = float(12)
>>> varA/varB
2.1665

And so on...

In fact, you only need to initialise one variable with float for this to
work:

>>> varA = float(26)
>>> varB = 12
>>> varA/varB
2.1665

This works in Python2 or Python3 without importing any extra libs.




--
James


On 12 June 2014 13:31, Steven D'Aprano  wrote:

> On Thu, Jun 12, 2014 at 08:48:25AM +0800, Marino David wrote:
> > Hi All:
> >
> > I am a newbie at the Python.
> >
> > I type "26/12" in Python Console and get result of "2".
> >
> > It is obvious that the corresponding result should be 2... I
> don't
> > know why the Console only returns the integer part of  true result.
> Anyone
> > can help me out?
>
> Try this instead:
>
> 26.0/12
>
> and it will print a fractional number instead of an int:
>
> py> 26.0/12
> 2.1665
>
>
> What's going on?
>
> Back in the early mists of time, when Python first came out, Python's
> creator Guido van Rossum decided that the / division operator should
> behave like in the C programming language. In C, division of two
> integers performs *integer division*, and drops the remainder, while
> division of one or more floating point number keeps the remainder as a
> fraction:
>
> 1/2 => 0
> 1/2.0 => 0.5
>
> That was one of those decisions that seemed like a good idea at the
> time, but turned out to be a mistake. But for backwards compatibility,
> Python had to keep it until recently.
>
> In Python version 3, / now does calculator division, like you expect.
> But in Python 2, you have to either convert one or both numbers to a
> float, or you can put this at the top of your program:
>
> from __future__ import division
>
> Note that there are TWO underscores at the beginning and end of
> "future".
>
>
> If you want integer division, where the remainder is ignored, you can
> use the // operator:
>
> py> 26.0//12
> 2.0
>
>
>
> --
> Steven
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] mixing 64 bit and 32 bit

[James Chapman]

Depending on what you're doing you could run into problems.

Areas that have been challenging for me in the past:
* Loading 64bit compiled .dll in a 32bit Python environment and vice versa.
* Reading registry entries. MS tried to be clever by introducing the
Wow6432Node reg key.

And I'm sure there has been a 3rd item that I am currently unable to
recall. It involved py2exe. This might have been related to the
registry issue mentioned above though.

With all that said, if you know you're likely to have these kinds of
issues it's pretty easy to write code that can deal with these types
of problems and do one thing or another depending on the base OS.




--
James


On 19 March 2014 19:53, John Fabiani  wrote:
> Thanks
> Johnf
>
> On 03/19/2014 11:01 AM, Reuben wrote:
>
> Hi John,
>
> The generated bytecodes will be different - but both version can run same
> code without issues
>
> Regards,
> Reuben
>
> On 19-Mar-2014 11:28 PM, "John Fabiani"  wrote:
>>
>> Hi,
>>
>> At my office we have a mix of XP (32bit) and Window 7 (64 bit).  I
>> installed python 64 bit on the windows 7 machines and 32 bit on the XP
>> machines.  The question is can the different version run the same code
>> without causing issues.  Can the 64 bit use the same byte code as the 32
>> bit?  It seems to be working but I thought best to check.
>>
>> Johnf
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] c++ on python

[James Chapman]

Perhaps I should look into Cython as I'm currently working on a
project that utilises a C API.

I've been finding that getting the data types to be exactly what the C
API is expecting to be the hardest part.

With the original question in mind, here's an example calling into a
C++ external C API:

(Works if compiled is VisualStudio. The DLL produced by MinGW-G++ didn't work).

---
// main.h

#ifndef __MAIN_H__
#define __MAIN_H__

#include 

#define DLL_EXPORT __declspec(dllexport)

#ifdef __cplusplus
extern "C"
{
#endif

int DLL_EXPORT add(int a, int b);

#ifdef __cplusplus
}
#endif

#endif // __MAIN_H__
---

---
//main.cpp

#include "main.h"

// a sample exported function
int DLL_EXPORT add(int a, int b)
{
return(a + b);
}

extern "C" DLL_EXPORT BOOL APIENTRY DllMain(HINSTANCE hinstDLL, DWORD
fdwReason, LPVOID lpvReserved)
{
switch (fdwReason)
{
case DLL_PROCESS_ATTACH:
// attach to process
// return FALSE to fail DLL load
break;

case DLL_PROCESS_DETACH:
// detach from process
break;

case DLL_THREAD_ATTACH:
// attach to thread
break;

case DLL_THREAD_DETACH:
// detach from thread
break;
}
return TRUE; // succesful
}
---

---
# -*- coding: utf-8 -*-
# dll.py

import ctypes


class DllInterface(object):

dll_handle = None

def __init__(self, dll_file):
self.dll_handle = ctypes.WinDLL(dll_file)

def add_a_and_b(self, a=0, b=0):
return self.dll_handle.add(a, b)


if __name__ == '__main__':
dll_file = 'PythonDLL.dll'
external_lib = DllInterface(dll_file)
int_a = ctypes.c_int(1)
int_b = ctypes.c_int(2)
result = external_lib.add_a_and_b(int_a, int_b)
print(result)

---
--
James
--
James


On 13 March 2014 15:57, Stefan Behnel  wrote:
> Alan Gauld, 12.03.2014 23:05:
>> On 12/03/14 16:49, Stefan Behnel wrote:
>>> Alan Gauld, 12.03.2014 10:11:
>>>> If it were a library then you would have to call
>>>> the individual C++ functions directly using
>>>> something like ctypes, which is usually more
>>>> complex.
>>>
>>> ctypes won't talk to C++, but Cython can do it quite easily.
>>
>> I thought it would work provided the interface functions
>> were declared as C functions? That might involve
>> writing a wrapper around it but that is usually
>> trivial if you have to compile the source anyway.
>
> The thing is: if you have to write your own wrapper anyway (trivial or
> not), then why not write it in Cython right away and avoid the intermediate
> plain C level?
>
> It's usually much nicer to work with object oriented code on both sides
> (assuming you understand the languages on both sides), than to try to
> squeeze them through a C-ish API bottleneck in the middle.
>
> Stefan
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When to use multiprocessing Managers?

[James Chapman]

Thanks for the explanation.

Is there any reason or case you can think of where on a single system
you would use the manager (proxied) queue over the multiprocessing
(piped) queue?

Actually I can probably answer this myself...

The manager could potentially be extended to do some kind of data
validation / error checking before submitting to the Queue. This could
be important if the data going into the Queue was for example, user
generated.

Hmm, yeah I'd say question answered. Thanks eryksun.


--
James


On 1 March 2014 16:48, eryksun  wrote:
> On Fri, Feb 28, 2014 at 6:31 AM, James Chapman  wrote:
>>
>> log_Q = multiprocessing.Queue()
>
> This is a Queue from multiprocessing.queues. It uses system resources
> (e.g. a semaphore for the queue capacity) that can be shared with
> processes on the same machine.
>
> A value `put` in a queue.Queue is available immediately:
>
> >>> import queue
> >>> q1 = queue.Queue()
> >>> try: q1.put('value'); q1.get_nowait()
> ... except queue.Empty: 'empty'
> ...
> 'value'
>
> On the other hand, a Queue from multiprocessing.queues writes to a
> pipe using a background thread, so there can be a small delay:
>
> >>> import multiprocessing as mp
> >>> q2 = mp.Queue()
> >>> try: q2.put('value'); q2.get_nowait()
> ... except queue.Empty: 'empty'
> ...
> 'empty'
> >>> q2.get_nowait()
> 'value'
>
>> or whether I create it like this:
>>
>> multimanager = multiprocessing.Manager()
>> log_Q = multimanager.Queue()
>
> This is a queue.Queue wrapped by an AutoProxy. For example, its `get`
> method calls _callmethod('get', *args, **kwds), which connects to the
> manager, sends the request, and receives the result.
>
> The docs demonstrate using a manager with remote processes:
>
> http://docs.python.org/3/library/multiprocessing#using-a-remote-manager
>
> You can also proxy the Queue type from multiprocessing.queues. In that
> case, remote processes use a proxy, but local processes can use the
> queue directly.
>
>> Perhaps the manager would be important if I was writing to a Queue and
>> expecting all threads to see that message?
>
> Only 1 thread will `get` the message.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python help

[James Chapman]

The answer lies in this page:
http://docs.python.org/3.3/library/stdtypes.html#string-methods

--
James


On 28 February 2014 11:44, James Chapman  wrote:
> The answer lies in this page:
> http://docs.python.org/3.3/library/stdtypes.html#string-methods
>
>
> --
> James
>
>
> On 28 February 2014 08:15, Ben Finney  wrote:
>> Tone Lite  writes:
>>
>>> I am having trouble coming up with a solution to this exercise and any
>>> help would be appreciated, thanks! I have attached the code below.
>>
>> The code you present appears to be the exercise. Are you asking for
>> someone to write the solution for you? That isn't what we do here. We'll
>> help you, but that doesn't mean we'll do your work for you.
>>
>> Please show us what you've tried, describe what it should be doing, and
>> what is happening instead. If there is an error, please also show the
>> complete error output.
>>
>> --
>>  \  “He who allows oppression, shares the crime.” —Erasmus Darwin, |
>>   `\ grandfather of Charles Darwin |
>> _o__)  |
>> Ben Finney
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When to use multiprocessing Managers?

[James Chapman]

Thanks for the replies so far...

As per the subject line, yes I'm talking about managers and any object
a manager is capable of spawning. This is not version specific or code
specific, it's more a discussion on when to use a manager and when
it's not needed.

>From the OReilly link sent by
Steven...(http://broadcast.oreilly.com/2009/04/pymotw-multiprocessing-part-2.html)
"The Manager is responsible for coordinating shared information state
between all of its users. By creating the list through the manager,
the list is updated in all processes when anyone modifies it. In
addition to lists, dictionaries are also supported."

In my case I have multiple threaded processing sending log messages to
a Queue that was created by the parent process. The parent process
then has a read thread to take messages out of the queue and write
them to file. Whether I create the queue object like this:

log_Q = multiprocessing.Queue()

or whether I create it like this:

multimanager = multiprocessing.Manager()
log_Q = multimanager.Queue()

seems to make no difference. I always get all the messages from all
the threads in all the processes.

Perhaps the manager would be important if I was writing to a Queue and
expecting all threads to see that message? Although if I needed to
command a thread to do something I'd probably have a separate class
and separate thread for that purpose.

James
--
James


On 26 February 2014 14:19, David Palao  wrote:
> 2014-02-25 11:52 GMT+01:00 James Chapman :
>> Hello tutors
>>
>> I'm curious about managers and when to use them.
>> For example, I see they offer a Queue() for sharing a Q between
>> processes, but if I create a Q in the parent process and pass it down
>> to child processes, then they can put messages into that Q just fine,
>> and I presume the same thing for other objects available under the
>> managers package.
>>
>> So unless the other process is on a different machine, is there a
>> reason to use a manager?
>>
>> Does anyone have any use case examples or snippets I could look at even?
>>
>> Thanks in advance
>> James
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>
> Hello,
> I asked myself the same question when I started using multiprocessing
> time ago. So I was very happy when I saw the question by James.
>
> From my limited knowledge, I would say that a Manager can be useful
> when processes are distributed across different hosts, or if the
> exchange of information between processes is more complex than just a
> couple of synchronization primitives.
>
> Best
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] When to use multiprocessing Managers?

[James Chapman]

Hello tutors

I'm curious about managers and when to use them.
For example, I see they offer a Queue() for sharing a Q between
processes, but if I create a Q in the parent process and pass it down
to child processes, then they can put messages into that Q just fine,
and I presume the same thing for other objects available under the
managers package.

So unless the other process is on a different machine, is there a
reason to use a manager?

Does anyone have any use case examples or snippets I could look at even?

Thanks in advance
James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I can't understand where python class methods come from

[James Scholes]

voger wrote:
> Some properties I can see them defined but some others like
> sublocality or administrative_area_level_1 I don't see them defined
> anywhere. Also in the comments in the source code the author says
> 
> #You can also choose a different property to display for each lookup
> #type. #Example: #result.country__short_name
> 
> but I can't understand where that __short_name comes from

The relevant code is in the __getattr__ method.  From the Python docs:

> If no class attribute is found, and the object’s class has a
> __getattr__() method, that is called to satisfy the lookup.
>
> -- Source:
> http://docs.python.org/2/reference/datamodel.html

As an aside, your examples are attributes, not methods.  Read the code
below; this is where the magic happens:

def __getattr__(self, name):
lookup = name.split('__')
attribute = lookup[0]

if (attribute in GeocoderResult.attribute_mapping):
attribute = GeocoderResult.attribute_mapping[attribute]

try:
prop = lookup[1]
except IndexError:
prop = 'long_name'

for elem in self.current_data['address_components']:
    if attribute in elem['types']:
return elem[prop]
-- 
James Scholes
http://twitter.com/JamesScholes
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Python .decode issue

[james campbell]

I have been currently trying to get a small piece of code to work, but keep 
getting an error:

header_bin = header_hex.decode('hex')
AttributeError: 'str' object has no attribute 'decode'


The source of this code is from: 
https://en.bitcoin.it/wiki/Block_hashing_algorithm
 Here is the the code:
  >>> import hashlib >>> header_hex = ("0100" + 
"81cd02ab7e569e8bcd9317e2fe99f2de44d49ab2b8851ba4a308" + 
"e320b6c2fffc8d750423db8b1eb942ae710e951ed797f7affc8892b0f1fc122b" + "c7f5d74d" 
+ "f2b9441a" + "42a14695") >>> header_bin = header_hex.decode('hex') >>> hash = 
hashlib.sha256(hashlib.sha256(header_bin).digest()).digest() >>> 
hash.encode('hex_codec') 
'1dbd981fe6985776b644b173a4d0385ddc1aa2a829688d1e' >>> 
hash[::-1].encode('hex_codec') 
'1e8d6829a8a21adc5d38d0a473b144b6765798e61f98bd1d'

I get the above error when I input this line by line into Win7 IDLE 3.3.3. 
Hopefully somebody can point me in the right direction. Thanks.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Unit testing infinite loops

[James Chapman]

Thanks Steven!

You've raised a few valid points, mostly that the run_forever method
should be broken up. I like the principle of a method doing just one
thing and for whatever reason I didn't apply that thinking to this
method as it's the master loop (even though it does nothing). So for
starters I'll fix that. Breaking everything up makes testing easier,
which in turn makes development easier.

On the note of what doesn't need a test...
The question of coverage always comes up when unit testing is
mentioned and I read an interesting blog article once about it. It
basically said: Assume you have 85% coverage on a program that
consists of 1,000 lines. That's 150 lines which are not tested. If
those lines are print lines, sleep lines, getters etc it's not really
a problem. But what happens when you scale that up. 1,000,000 lines of
code lets say (not unheard of, although in python that would be out of
this world big). You now end up with 150,000 lines of untested code.
While the percentage of code covered is high, there is _a_lot_ of code
there that isn't tested and a lot of room for mistakes to creep in. A
mistake on one of those 150,000 lines could break the build and
possibly cost you hours or even days tracking it down. If those lines
were tested however, your continuous integration build system would
hopefully highlight the fault.

In my experience testing works, saves time down the line, and makes
code easier to come back to.
--
James


On 31 January 2014 13:21, Steven D'Aprano  wrote:
> On Fri, Jan 31, 2014 at 11:31:49AM +, James Chapman wrote:
>> Hello tutors
>>
>> I've constructed an example which shows a problem I'm having testing a real
>> world program and would like to run it past you.
> [...]
>> class Infinite_Loop_Tutor_Question(object):
>> def run_forever(self):
>> self.start_A()
>> time.sleep(0.5)
>> self.start_B()
>> try:
>> while True:
>> time.sleep(1)
>> except KeyboardInterrupt:
>> print("Caught Keyboard Interrupt...")
>> sys.exit(0)
> [...]
>> In my example above, testing the everything but the run_forever method is
>> trivial.
>>
>> So on to my question... The run_forever method essentially just fires up a
>> bunch of threads to serve various purposes and then waits for CTRL-C to
>> terminate the entire program. Testing this at the moment is very difficult
>> because the unit test ends up in the infinite loop. So, would a better idea
>> be to create an attribute, set it to True and then do
>>
>> try:
>> while self.attribute:
>> time.sleep(1)
>> except KeyboardInterrupt:
>> ...
>
>
> That probably won't hurt.
>
>
>> My unit test could then set the attribute. However I'd still have the
>> problem of how I get from the unit test line that fires up the method to
>> the next line to change the attribute.
>>
>> So how should the run_forever method be written so that it's testable, or
>> if it's testable as is, how would I test it?
>
> What are you trying to test? You don't just "test" a method, you test
> *something specific* about the method. So what specifically are you
> trying to test?
>
>
>> And please, no comments about syntax, clean exits of threads, thread
>> communication, resources, or even the need for testing the run_forever
>> method. In my test I want to test that it makes the relevant calls and then
>> enters the infinite loop at which point I want to terminate it.
>
> Ah, you see, now you have a problem. Consider this function:
>
> def long_calc():
> time.sleep(60*60*24*365)
> return 1
>
>
> How do I test that the function returns 1? As given, I can't
> really, not unless I wait a whole year for the sleep() to return. So
> what I can do is split the function into two pieces:
>
> def _sleep_a_year():
> time.sleep(60*60*24*365)
>
> def _do_calculation():
> return 1
>
> def long_calc():
> _sleep_a_year()
> return _do_calculation()
>
>
> Now I can unit-test the _do_calculation function, and long_calc() is now
> simple enough that I don't really need to unit-test it. (Unit testing
> should not be treated as a religion. You test what you can. Any testing
> is better than nothing, and if there are some parts of the program which
> are too hard to test automatically, don't test them automatically.)
>
> Or, I can monkey-patch the time.sleep function. Before running my
> test_long_calc unit-test, I do this:
>
> import time
> time.sleep = lambda n: None
>
&

Re: [Tutor] Unit testing infinite loops

[James Chapman]

Hmm...

Here is an example of how I'm currently trying to test it:

test_tutor_question.py
-
# -*- coding: utf-8 -*-
import unittest
import mock
from tutor_question import Infinite_Loop_Tutor_Question


class Test_Infinite_Loop_Tutor_Question(unittest.TestCase):

def test_run_forever(self):
with mock.patch('tutor_question.Infinite_Loop_Tutor_Question.start_A')
as start_A:
with
mock.patch('tutor_question.Infinite_Loop_Tutor_Question.start_B') as
start_B:
inf_loop = Infinite_Loop_Tutor_Question()
print start_A.call_count
print start_B.call_count
inf_loop.run_forever()
inf_loop.interrupt_main()
print start_A.call_count
print start_B.call_count


if __name__ == "__main__":
unittest.main()
-

As you can see if you run this, the test doesn't reach the lines below
inf_loop.run_forever().

So ideally, I'd need a way of injecting a keyboard interrupt into the
method, I could then check that the exception was handled and that the
start_A and start_B calls were made.

** Obviously the print lines will be substituted for some kind of
assert lines **

FYI I'm using CPython 2.7.



--
James


On 31 January 2014 12:57, eryksun  wrote:
>
> On Fri, Jan 31, 2014 at 6:31 AM, James Chapman  wrote:
> > try:
> > while self.attribute:
> > time.sleep(1)
> > except KeyboardInterrupt:
> > ...
> >
> > My unit test could then set the attribute. However I'd still have the
> > problem of how I get from the unit test line that fires up the method to the
> > next line to change the attribute.
>
> You could add a method that toggles the attribute, and use a
> threading.Timer to run it after a set interval.
>
> > if it's testable as is, how would I test it?
>
> CPython 2.3+ can interrupt the main thread from another thread using
> the built-in function `_thread.interrupt_main`:
>
> http://docs.python.org/3/library/_thread#_thread.interrupt_main
>
> >>> import _thread
> >>> _thread.interrupt_main()
> Traceback (most recent call last):
>   File "", line 1, in 
> KeyboardInterrupt
>
> It's also implemented in PyPy, but not in Jython.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Unit testing infinite loops

[James Chapman]

Hello tutors

I've constructed an example which shows a problem I'm having testing a real
world program and would like to run it past you.

tutor_question.py
--
# -*- coding: utf-8 -*-
import sys
import threading
import time


class Time_Printer(threading.Thread):

def run(self):
for i in range(60):
print('%s - %s' % (self, time.ctime(time.time(
time.sleep(1)


class Infinite_Loop_Tutor_Question(object):

def start_A(self):
thread = Time_Printer()
thread.daemon = True
thread.start()
print('Started %s' % (thread))

def start_B(self):
thread = Time_Printer()
thread.daemon = True
thread.start()
print('Started %s' % (thread))

def run_forever(self):
self.start_A()
time.sleep(0.5)
self.start_B()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
print("Caught Keyboard Interrupt...")
sys.exit(0)


if __name__ == '__main__':
infinite_loop = Infinite_Loop_Tutor_Question()
infinite_loop.run_forever()

--

In my example above, testing the everything but the run_forever method is
trivial.

So on to my question... The run_forever method essentially just fires up a
bunch of threads to serve various purposes and then waits for CTRL-C to
terminate the entire program. Testing this at the moment is very difficult
because the unit test ends up in the infinite loop. So, would a better idea
be to create an attribute, set it to True and then do

try:
while self.attribute:
time.sleep(1)
except KeyboardInterrupt:
...


My unit test could then set the attribute. However I'd still have the
problem of how I get from the unit test line that fires up the method to
the next line to change the attribute.

So how should the run_forever method be written so that it's testable, or
if it's testable as is, how would I test it?

And please, no comments about syntax, clean exits of threads, thread
communication, resources, or even the need for testing the run_forever
method. In my test I want to test that it makes the relevant calls and then
enters the infinite loop at which point I want to terminate it.

Thanks in advance, and hopefully there are no formatting issues this time.


--
James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Mocking with "mock" in unit testing

[James Chapman]

Really!

import mock
import unittest
import pinger

It should be three lines, but somehow it got all messed up, either
through rich text formatting or copy paste.

Being a bit pedantic now about import statements which are clearly
unintentionally messed up.

- Sent in plain text.



--
James




On 17 January 2014 15:32, Steven D'Aprano  wrote:
>
> On Fri, Jan 17, 2014 at 08:35:04AM -0500, eryksun wrote:
> > On Fri, Jan 17, 2014 at 6:23 AM, Steven D'Aprano  
> > wrote:
> > > On Fri, Jan 17, 2014 at 09:58:06AM +, James Chapman wrote:
> [...]
> > >> import mockimport unittestimport pinger
> > >>  class Test_Pinger(unittest.TestCase):
> > >
> > > And here you have two more SyntaxErrors: missing commas between
> > > arguments to import, and a stray space before the "class" again.
> >
> > This is also fine in the rich text version. BTW, the botched plain
> > text conversion is missing line breaks, not commas.
>
> I don't understand this. If I'm interpreting you correctly,
>
>
> import mockimport
> unittestimport
> pinger
>
>
> would not be "fine".
>
>
> --
> Steven
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Mocking with "mock" in unit testing

[James Chapman]

Ah of course!
pinger = pinger.Pinger()  I should have noticed that myself.  (I
renamed the file to pinger.py after establishing it all worked and then
didn't re-run).
ping = pinger.Pinger() works fine.

As for the syntax errors, those will be copy paste / different email client
errors.

--
James


On 17 January 2014 11:23, Steven D'Aprano  wrote:

> On Fri, Jan 17, 2014 at 09:58:06AM +, James Chapman wrote:
>
> > As this question was just about mock and not really dealing with the bad
> > return code or exception handling or raising my final working example
> looks
> > like this:
>
> Your final *working* example? I don't think so. I can see at least two
> syntax errors, and a programming error.
>
> A word to the wise: code isn't working until you've actually seen it
> work :-)
>
>
>
> > pinger.py
> >
> > 
> >
> > import subprocess
> >  class Pinger(object):
>
> There is your first SyntaxError, a stray space ahead of the "class"
> keyword.
>
>
> > def ping_host(self, host_to_ping):
> > cmd_string = 'ping %s' % (host_to_ping)
> > cmd_args = cmd_string.split()
>
> This is not a programming error, but it is wasteful. First you join two
> strings: "ping", and the host. Then, having glued them together, you
> split them apart exactly where you glued them.
>
> Better to do something like:
>
> cmd_args = ["ping", host_to_ping]
>
> assuming that you know that host_to_ping is only a single word.
>
>
> > proc = subprocess.Popen(cmd_args, shell=True)
>
> Why shell=True?
>
> The documentation for subprocess.Popen says:
>
> The shell argument (which defaults to False) specifies whether to
> use the shell as the program to execute. If shell is True, it is
> recommended to pass args as a string rather than as a sequence.
>
> and also warns that shell=True can be a security hazard. Do you have a
> good reason for using it?
>
> http://docs.python.org/2/library/subprocess.html
>
>
>
> > proc.wait()
> > if proc.returncode != 0:
> > raise Exception('Error code was: %d' % (proc.returncode))
> >
> >  if __name__ == '__main__':
> > PINGER = Pinger()
> > PINGER.ping_host('localhost')
> >
> > 
> >
> >
> >
> > test_pinger.py
> >
> > 
> >
> > import mockimport unittestimport pinger
> >  class Test_Pinger(unittest.TestCase):
>
> And here you have two more SyntaxErrors: missing commas between
> arguments to import, and a stray space before the "class" again.
>
>
> > def test_ping_host_succeeds(self):
> > pinger = pinger.Pinger()
>
> And here is your programming error. Unlike some programming languages
> (Lua, I think) in Python you cannot use the same name to refer to both a
> global variable and a local variable inside the same function. In
> Python, either the variable is treated as a global, or as a local, but
> not both.
>
> The rules Python uses to decide are:
>
> * if you declare a name "global" inside the function, then it is
>   treated as global inside that function;
>
> * otherwise, if you assign a value to the name *anywhere* inside
>   the function, it is treated as local;
>
> * otherwise it is global.
>
> So in your example, you assign a value to the name "pinger", so it is
> treated as local. The offending line of code looks like this:
>
> pinger = pinger.Pinger()
>
> Your *intention* is to look up the global "pinger", which is a module,
> then create a pinger.Pinger() instance, then assign it to the local
> variable "pinger". But that's not what Python does. Instead, it tries to
> look up a local variable "pinger", finds it doesn't have one, and raises
> an UnboundLocalError exception.
>
> I recommend you change the name of the local variable "pinger" to
> something else, so it no longer clashes with the "pinger" module name.
>
>
> Hope this is of help to you,
>
>
>
> --
> Steven
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Mocking with "mock" in unit testing

[James Chapman]

Erm...?

CPython yeah.

If I rename "pinger.py" to "tutor.py" and change the unittest it runs fine.
Why?


-
import mock
import unittest
import tutor

class Test_Pinger(unittest.TestCase):

def test_ping_host_succeeds(self):
pinger = tutor.Pinger()
with mock.patch("tutor.subprocess") as subprocess:
subprocess.Popen.return_value.returncode = 0
pinger.ping_host('localhost')
subprocess.Popen.assert_called_once_with(['ping','localhost'],
shell=True)


def test_ping_host_fails_and_throws_exception(self):
pinger = tutor.Pinger()
with mock.patch('tutor.subprocess') as subprocess:
subprocess.Popen.return_value.returncode = 1
self.assertRaises(Exception, pinger.ping_host, 'localhost')



if __name__ == '__main__':
unittest.main()
-


--
James


On 17 January 2014 10:50, eryksun  wrote:

> On Fri, Jan 17, 2014 at 4:58 AM, James Chapman 
> wrote:
> > import mock
> > import unittest
> > import pinger
> >
> > class Test_Pinger(unittest.TestCase):
> >
> > def test_ping_host_succeeds(self):
> > pinger = pinger.Pinger()
>
> Are you using CPython? That raises an UnboundLocalError. Take a look
> at the CPython bytecode:
>
> def test(self):
> pinger = pinger.Pinger()
>
> >>> dis.dis(test)
>   2   0 LOAD_FAST1 (pinger)
>   3 LOAD_ATTR0 (Pinger)
>   6 CALL_FUNCTION0
>   9 STORE_FAST   1 (pinger)
>  12 LOAD_CONST   0 (None)
>  15 RETURN_VALUE
>
> Notice LOAD_FAST(1), where fast local 1 is the local variable
> `pinger`. The value isn't assigned yet, and LOAD_FAST doesn't search
> globals and builtins. You need a unique name for the instance, such as
> `test_pinger`. Then the compiler knows to use LOAD_GLOBAL:
>
> def test(self):
> test_pinger = pinger.Pinger()
>
> >>> dis.dis(test)
>   2   0 LOAD_GLOBAL  0 (pinger)
>   3 LOAD_ATTR1 (Pinger)
>   6 CALL_FUNCTION0
>   9 STORE_FAST   1 (test_pinger)
>  12 LOAD_CONST   0 (None)
>  15 RETURN_VALUE
>
> Here's a version using a decorator instead:
>
> @mock.patch('pinger.subprocess')
> def test_ping_host_succeeds(self, subprocess):
> subprocess.Popen.return_value.returncode = 0
> test_pinger = pinger.Pinger()
> test_pinger.ping_host('localhost')
> subprocess.Popen.assert_called_once_with(['ping','localhost'],
>  shell=True)
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Mocking with "mock" in unit testing

[James Chapman]

Thanks eryksun.

There's a list for testing in python which I also posed the question to and
got pretty much the same answer as you provided.

The line if proc.returncode != 1 was a mistake. That 1 should have been a
zero.

As this question was just about mock and not really dealing with the bad
return code or exception handling or raising my final working example looks
like this:


pinger.py



import subprocess
 class Pinger(object):

def ping_host(self, host_to_ping):
cmd_string = 'ping %s' % (host_to_ping)
cmd_args = cmd_string.split()
proc = subprocess.Popen(cmd_args, shell=True)
proc.wait()
if proc.returncode != 0:
raise Exception('Error code was: %d' % (proc.returncode))

 if __name__ == '__main__':
PINGER = Pinger()
PINGER.ping_host('localhost')





test_pinger.py



import mockimport unittestimport pinger
 class Test_Pinger(unittest.TestCase):

def test_ping_host_succeeds(self):
pinger = pinger.Pinger()
with mock.patch("pinger.subprocess") as subprocess:
subprocess.Popen.return_value.returncode = 0
pinger.ping_host('localhost')
subprocess.Popen.assert_called_once_with(['ping','localhost'],
shell=True)

def test_ping_host_fails_and_throws_exception(self):
pinger = pinger.Pinger()
with mock.patch('pinger.subprocess') as subprocess:
subprocess.Popen.return_value.returncode = 1
self.assertRaises(Exception, pinger.ping_host, 'localhost')

 if __name__ == '__main__':
unittest.main()

----




--
James


On 17 January 2014 01:05, eryksun  wrote:

> On Thu, Jan 16, 2014 at 5:32 AM, James Chapman 
> wrote:
> >
> > In my unittest I don't want to run the ping command, (It might not be
> > available on the build system) I merely want to check that a call to
> > subprocess.Popen is made and that the parameters are what I expect?
>
> You can mock `Popen` where it's accessed.
>
> @mock.patch('subprocess.Popen')
> def test_ping_host(self, Popen):
> Popen.return_value.returncode = 0
> pinger = Pinger()
> pinger.ping_host('127.0.0.1')
> Popen.assert_called_once_with(['ping','127.0.0.1'], shell=True)
>
> If the tutor_q module imported `Popen` into its namespace, then you'd
> patch tutor_q.Popen.
>
> > def ping_host(self, host_to_ping):
> > cmd_string = 'ping %s' % (host_to_ping)
> > cmd_args = cmd_string.split()
>
> Splitting on spaces doesn't work generally. Use `shlex.split`, or
> build the list manually.
>
> > proc = subprocess.Popen(cmd_args, shell=True)
>
> Maybe you really need the shell to process your command, but generally
> there's no reason to run the shell just to have it execute the command
> and wait. Plus it opens the door to security exploits.
>
> > proc.wait()
> > if proc.returncode != 1:
> > raise Exception('Error code was: %d' % (proc.returncode))
>
> A non-zero return code signals an error. When using `Popen` directly,
> you can be consistent with `check_call` and `check_output` if you
> raise a `CalledProcessError` in this case:
>
> retcode = proc.wait()
> if retcode:
> raise subprocess.CalledProcessError(retcode, cmd_args)
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Mocking with "mock" in unit testing

[James Chapman]

Hi all

I have a question regarding mocking in unit testing.

Let's assume I have the following class:

---
import subprocess

class Pinger(object):

def ping_host(self, host_to_ping):
cmd_string = 'ping %s' % (host_to_ping)
cmd_args = cmd_string.split()
proc = subprocess.Popen(cmd_args, shell=True)
proc.wait()
if proc.returncode != 1:
raise Exception('Error code was: %d' % (proc.returncode))

---


In my unittest I don't want to run the ping command, (It might not be
available on the build system) I merely want to check that a call to
subprocess.Popen is made and that the parameters are what I expect?

So far I have this, but it doesn't work and I suspect it's way off!!


---
import mock
import unittest
from tutor_q import Pinger

class Test_Pinger(unittest.TestCase):

def test_ping_host(self):
pinger = Pinger()
assert pinger
subprocess = mock.Mock()
subprocess.Popen.return_value = 0
subprocess.assert_called_once_with(['ping','localhost'])
pinger.ping_host('127.0.0.1')




if __name__ == '__main__':
unittest.main()

---


Can anyone point me in the right direction on how to mock up these
subprocess calls?

Thanks

James
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] i installed the package statsmodels but i get an error when i use it

[Derry, James R]

hi, eva,

when i ran your code i found a misspell:
result = statsmodels.tsa.statools.adfuller(x,1)

when i fixed 'stattools' --
result = statsmodels.tsa.stattools.adfuller(x,1)

print result

(-2.6825663173365015, 0.077103947319183241, 0, 7, {'5%': -3.4775828571428571, 
'1%': -4.9386902332361515, '10%': -2.8438679591836733}, 15.971188911270618)

-- james


From: Tutor [tutor-bounces+jderry=mail.utexas@python.org] on behalf of eva 
maria gualandi [evamaria.guala...@gmail.com]
Sent: Monday, December 16, 2013 10:56 AM
To: tutor@python.org
Subject: [Tutor] i installed the package statsmodels but i get an error when
i use it

Good afternoon,
I installed from https://pypi.python.org/pypi/statsmodels the package 
statsmodels for python 2.7 (32bit) , i have to do some test statistics for a 
time series analysis that i can find under statsmodels.tsa. In particular i 
really need to do a Dickey Fuller test and in agreement with the statsmodels 
documentation ( from 
http://statsmodels.sourceforge.net/devel/generated/statsmodels.tsa.stattools.adfuller.html
 ) i run this simple programm

import numpy as np
import statsmodels
from statsmodels.tsa import stattools
x = np.array([1,2,3,4,3,4,2,3])
result = statsmodels.tsa.statools.adfuller(x,1)
print result

but unfortunately i get the following error message:

Traceback (most recent call last):
  File "C:\Python27\esempio_adftest.py", line 3, in 
from statsmodels.tsa import stattools
  File "C:\Python27\lib\site-packages\statsmodels\tsa\stattools.py", line 7, in 

from statsmodels.regression.linear_model import OLS, yule_walker
  File "C:\Python27\lib\site-packages\statsmodels\regression\__init__.py", line 
1, in 
from linear_model import yule_walker
  File "C:\Python27\lib\site-packages\statsmodels\regression\linear_model.py", 
line 37, in 
from statsmodels.tools.tools import (add_constant, rank,
  File "C:\Python27\lib\site-packages\statsmodels\tools\__init__.py", line 1, 
in 
from tools import add_constant, categorical
  File "C:\Python27\lib\site-packages\statsmodels\tools\tools.py", line 14, in 

from pandas import DataFrame
  File "C:\Python27\lib\site-packages\pandas\__init__.py", line 15, in 
raise ImportError('C extensions not built: if you installed already '
ImportError: C extensions not built: if you installed already verify that you 
are not importing from the source directory

the thing that i really don't understand is that in the shell window if i type 
import statsmodels.tsa i don't get any error, also if i type the help funcion i 
can see that in the package there is the function stattools that i need

>>> import statsmodels.tsa
>>> help(statsmodels.tsa)
Help on package statsmodels.tsa in statsmodels:

NAME
statsmodels.tsa

FILE
c:\python27\lib\site-packages\statsmodels\tsa\__init__.py

PACKAGE CONTENTS
adfvalues
api
ar_model
arima_model
arima_process
arma_mle
base (package)
descriptivestats
filters (package)
interp (package)
kalmanf (package)
mlemodel
setup
stattools
tests (package)
tsatools
varma_process
vector_ar (package)
I will really appreciate your help
thanks in advance,

Eva

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] why do i keep getting syntax errors in python when this runs

[James Griffin]

!-- On Fri  6.Sep'13 at  5:27:23 BST, mike johnson (pretor...@hotmail.com), 
wrote: 
> can you please help me figure out why this isnt working thanks
> # convert.py
> # this program is used to convert Celsius temps to Fahrenheit
> # By: James Michael Johnson
> 
> Def main ():
> Celsius = float (input ("What is the Celsius temperature? "))
> Fahrenheit = 9.0 / 5.0 * Celsius + 32
> Print ("The temperature is ", Fahrenheit, " degrees Fahrenheit.")
> 
> 
> Main ()

[ ...]

> # convert.py
> # this program is used to convert Celsius temps to Fahrenheit
> # By: James Michael Johnson
> 
> Def main ():
> Celsius = float (input ("What is the Celsius temperature? "))
> Fahrenheit = 9.0 / 5.0 * Celsius + 32
> Print ("The temperature is ", Fahrenheit, " degrees Fahrenheit.")
> 
> 
> Main ()

I'm sure this is homework. I had a peice of homework in my first year
Undergrad exactly like this. 

-- 


James Griffin: jmz at kontrol.kode5.net 

A4B9 E875 A18C 6E11 F46D  B788 BEE6 1251 1D31 DC38
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can't install latest PIL

[James Reynolds]

You may want to consider pillow. Oil hasn't been maintained in some time.
 On May 16, 2013 6:12 PM, "Jim Mooney"  wrote:

> Make sure you have the correct architecture. The builds from
> PythonWare are 32-bit. Christoph Gohlke has 64-bit builds here:
>
> http://www.lfd.uci.edu/~gohlke/pythonlibs/#pil
>
> ==
> Okay, I installed the 64 bit for Py 2.7, it installed, I see  PIL
> directory in Site-Packages. (Actually, it's the Pillow fork, but it
> still seems to be named PIL)  I tried "import image from pil" and got
> the following error: SyntaxError   invalid syntax (python_init.py,
> line 1)
>
> I don't see where that prog is, so it's an install somewhere and I
> don't want to mess with it anyway.
>
> Oh well, be nice to get back to learning Python after wrestling with
> editors, installs, and Windows ;')
>
> Jim
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] building a website with python

[James Reynolds]

As far as hosts, I use digital ocean. It's a cloud based thing like EC2,
but it's cheap (5/10/20 and up). You will of course have to configure
everything yourself, but that's not such a bad thing.

It will give you good experience configuring a linux box as well and
learning about deployment.

Since I'm a django developer, I'm going to suggest you learn that of
course. But, there are others out there. However, with Django you get the
Django eco system, which is large and very helpful.

For example, you can use fabric for easy deployment, south for database
migrations, celery for task management and so and so forth.

Even simple websites could find some use in south and fabric.


On Tue, Apr 9, 2013 at 6:06 PM, Walter Prins  wrote:

> Hello Benjamin,
>
> On 9 April 2013 22:31, Benjamin Fishbein  wrote:
>
>> Hello. I learned Python this past year (with help from many of you) and
>> wrote many programs for my small business. Now I want to build a website. I
>> acquired the domain name through godaddy.com (bookchicken.com) but have
>> not found hosting yet.
>> I learned html, css, and javascript via codeacademy.org, but have never
>> built a website.
>> I would like to build it with Python, and was wondering if you could give
>> me some pointers about what I need to learn first: Django is the thing to
>> use? And do I need specific hosting to use python or are all hosts
>> basically the same.
>>
>
> Django is good but arguably big and a bit intimidating if you're starting
> out.  There are many alternatives, from so called "micro" frameworks like
> "bottle" (which is implemented in a single Python file) to CherryPy to
> Flask.  Personally, I'd suggest having a look at Web2Py.  It's not quite as
> heavy as Django but very capable nonetheless.  It also deploys easily,
> including to Google App Engine and elsewhere, and includes a very capable
> Admin interface that also includes a basic IDE, thus allowing you to tweak
> the site right from within the site should you have the need. Here's a nice
> little presentation about Web2Py:
> http://www.slideshare.net/blackthorne/web2pyweb-development-like-a-boss
> ... and a tutorial to get you going:
> http://killer-web-development.com/section/1/0
>
> Not all web hosts are the same -- you need to ensure you have Python
> available with the host you use.  There are dedicated Python hosts out
> there.  (PythonAnywhere comes to mind.)  As a slightly different
> possibility, you might consider running your own virtual host in the cloud.
>  You would need to learn a bit about administering a server box, but this
> route does give you the ultimate control over your website and environment.
>  Here's a link to Rackspace with which I've had good experiences thus far
> for your review:
> http://www.rackspace.com/cloud/servers/
>
> Walter
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python on ipad

[James Griffin]

[- Sun 10.Mar'13 at 16:42:59 -0500  Benjamin Fishbein :-]

> Hello. I wrote some python programs for my small business that I run on my 
> computer...macbook air. I'm planning to backpack around Mexico and perhaps 
> south america. I'll still be working though. Basically my computer does all 
> the work, I just need to have internet connections and run the programs, and 
> periodically click here and there.
> I don't want to take my macbook with me because I'd have anxiety that it'd 
> get stolen and I wouldn't have any fun.
> So I'm debating if I should get a cheap computer for a couple hundred bucks 
> and run the python scripts on it. I think this is possible because I hear the 
> code is the same whether it's mac or PC or whatever.
> Or I might take my ipad with me. Or just run it on my iphone.
> Do you know if it's possible to run python scripts on a ipad/iphone, and if 
> so how to do it?
> Thanks,
> Ben

An iPad will be far more atttractive to a thief than a laptop, Mac or 
otherwise. Just get a cheap laptop and put linux on it. You can use your python 
programs on that easily enough. 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Which pip for Ubuntu 12.04

[James Reynolds]

The bigger issue with mongo is the apt versions are old. Be sure to follow
the instructions on mongos site.

If you pip install pymongo with a ubunuto or mint build your gtg
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] First Python Test

[James Griffin]

--> Shall, Sydney  [2013-02-02 12:45:15 +]:

> Two free good text editors for the MAC are;
> 1. Komodo
> 2. Text Wrangler.
> hth
> Sydney

aquamacs is the correct gui version to use. There is also a gvim
binary available. Both emacs and vim are installed already on Mac
OS X but in text only versions, no GUI.

I have never used Komodo or Text Wrangler, or any other GUI editor
for that matter so I cannot comment on how good they are. I tend
to spend about 90% of my time using the command-line on my Mac, but
that's just my preference. Some people prefer GUI editors.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] How can I overwrite the previous version on linux ?

[Dae James]

My linux distribution is CentOs 6.3. And python attached to the OS is 2.6. 
How can I overwrite the previous version with python 2.7 ? Or how can I 
uninstall the previous version?




Dae James___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PostreSQL

[James Reynolds]

Psycopg2 is the driver for postgres. Not sure if it is py3k compliant
On Nov 15, 2012 6:13 AM, "Válas Péter"  wrote:

> Two listings, of course. :-)
> I failed to tell that my PG version is "PostgreSQL 8.1.2" (and my client
> uses Windows 7, if this has relevance).
>
> 2012/11/15 Válas Péter 
>
>> Hi folks,
>>
>> Here are to listings of Python interfaces to PostgreSQL:
>> Postgres wiki: http://wiki.postgresql.org/wiki/Python
>> Python wiki: http://wiki.python.org/moin/PostgreSQL
>>
>> I have Python 3.2.3.
>> Does anybody have experiences with these modules? Which is worth to use?
>> Any point of view to compare them?
>>
>> Thanks, Péter
>>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Why VPython can't be searched out in PyPI?

[Dae James]

I found that VPython is not in PyPI(python packet index from www.python.org). 
Why ?




Dae James___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Question about language code

[Dae James]

Here is a example in "Python v2.7.2 document":

>>> import locale
>>> loc = locale.getlocale() # get current locale
# use German locale; name might vary with platform
>>> locale.setlocale(locale.LC_ALL, 'de_DE')

However, the result of executing on my computer is:
>>> locale.setlocale(locale.LC_ALL, 'de_DE')
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\locale.py", line 531, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting

Why is it unsuccessful? 
My os is Windows 7. My python version is 2.7.2. 




Dae James___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What can I do if I'm banned from a website??

[James Reynolds]

On Wed, Oct 10, 2012 at 4:35 PM, Benjamin Fishbein wrote:

> I've been scraping info from a website with a url program I wrote. But now
> I can't open their webpage, no matter which web browser I use. I think
> they've somehow blocked me. How can I get back in? Is it a temporary block?
> And can I get in with the same computer from a different wifi?
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

A few thoughts:

1. Try using a proxy.
2. Ask the webadmin (nicely) to unban you
3. Use requests / urllib3 to se connection pooling
4. See if the site has an API designed for data extraction.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] new student

[james gillis]

I am looking for some recommendations books to
read.websites,links,any information would be appreciated. Thanks,
jmslgil...@gmail.com
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] How can I convert a variable name to a string?

[Dae James]

How can I convert a variable name to a string ? 
For example:
testVariable = 1000;
How can I get the string "testVariable" ?
Thank you~




Dae James___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python cms

[James Reynolds]

I agree with Steven.

You will notice efficiency gains by moving to python.

I'm a django developer myself, but some of the other frameworks work well also.

That said the django community is fairly large, and there is an active cms 
project called django-cms.

https://www.django-cms.org/



Sent from my iPad

On Aug 19, 2012, at 3:10 AM, Steven D'Aprano  wrote:

> On 19/08/12 15:05, Douglas Kuo wrote:
>> Hi all,
>> 
>> I am mainly a php/drupal developer totally new to python.. Any experienced
>> python especially in term of cms developers out there can tell me what are
>> the benefits of python cms over php cms?
> 
> This might upset you, but the biggest advantage is that they aren't PHP.
> 
> http://www.codinghorror.com/blog/2012/06/the-php-singularity.html
> 
> http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/
> 
> I know what it's like to have people dissing your favourite language and
> telling you it isn't a "proper" programming language. But you know, sometimes
> there are good reasons why they do that, not just fashion.
> 
> I found this spirited defense of PHP here:
> 
> http://blog.ircmaxell.com/2012/04/php-sucks-but-i-like-it.html
> 
> which raises some good points, but I think when the author finds himself
> defending PHP's == operator in the comments by saying:
> 
> "It may not be sane, but it works..."
> 
> then I think he's just conceded the argument.
> 
> 
> The biggest disadvantage of using a Python CMS is that if you're not running
> your own web server, you might have to call up two or three ISPs to find one
> that will provide you with Python on the server.
> 
> In the Python ecosystem, we have Zope, which is a big, powerful heavyweight
> CMS. For a beginner, you might prefer to start with something more
> lightweight. Perhaps try Django:
> 
> https://www.djangoproject.com/
> 
> See also http://wiki.python.org/moin/ContentManagementSystems
> 
> 
> For a wiki, try MoinMoin:
> 
> http://moinmo.in/
> 
> For web frameworks, try CherryPy:
> 
> http://www.cherrypy.org/
> 
> 
> 
> -- 
> Steven
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I-Phone App in Python?

[James Reynolds]

the "native app" part of building an "app" is somewhat trivial in
comparison to the backend needs.

Read this article here on what instagram uses for their architecture:
http://instagram-engineering.tumblr.com/post/13649370142/what-powers-instagram-hundreds-of-instances-dozens-of


As you can see, most of it is Python libraries, some of which are django
specific, some of which aren't (note, theya are using custom written task
system instead of using Celery - not sure why this is).

As I understand it, interpreted languages on phones simply take up too much
space and memory.

I know some companies have used
http://www.appcelerator.com/<http://www.appcelerator.com/developers>
with
good success

That said, unless all you want to your app to do is only interact with the
user from within the phone and never get information from outside of it or
send information outside of it, then I suppose you won't be able to use
Python to any degree.

But, if you do, then you can write fantastic apps using python libraries
for the backend needs.

On Sat, Aug 4, 2012 at 2:28 PM, Chris Fox  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 04/08/2012 19:18, James Reynolds wrote:
> > Instagram is written in python django.
> >
>
> Even if that were true (Django is, as I understand it, a web
> framework) - how does that information help the original poster?
>
> Chris
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAlAdabcACgkQ/UgLKfoJxI5f9QCgwA1LR2S0EqUpxWqlFaGE9Dl6
> yGEAoLqYlTDn6kZWrZ+PN20uqpmsSbPJ
> =Oeru
> -END PGP SIGNATURE-
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I-Phone App in Python?

[James Reynolds]

To clarify, that is their server side arc. The native app itself...  I'm not 
sure what is written in. Probably java if I had to guess.

In making an app, most of your work is going to be backend in any event.

Also, what happens when an android user wants to download your app? 

Sent from my iPad

On Aug 3, 2012, at 7:13 PM, Fred G  wrote:

> I just googled whether it is possible to write an i-phone app in Python and 
> got very confusing, and not super good results.
> 
> Is it possible? And if so, what module(s) do I need to install?
> 
> Much thanks!
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I-Phone App in Python?

[James Reynolds]

Instagram is written in python django. 

Sent from my iPad

On Aug 3, 2012, at 7:13 PM, Fred G  wrote:

> I just googled whether it is possible to write an i-phone app in Python and 
> got very confusing, and not super good results.
> 
> Is it possible? And if so, what module(s) do I need to install?
> 
> Much thanks!
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] advice on global variables

[James Reynolds]

On Wed, Jul 11, 2012 at 10:30 AM, Chris Hare  wrote:

>
> On Jul 11, 2012, at 8:05 AM, Walter Prins wrote:
>
> > [snip]
>
> > Your original example modified as demonstration:
> >
> > a.py:
> > 
> > import shared
> > import b
> >
> > def func1():
> >print "global var in func1 = %s" % shared.global_var
> >
> > class intclass:
> >def func2(self):
> >print "global var in intclass = %s" % shared.global_var
> >
> > print "global_var = %s" % shared.global_var
> > func1()
> > f = intclass()
> > f.func2()
> > g = b.extclass()
> > g.func3()
> >
> > b.py:
> > 
> >
> > import shared
> >
> > class extclass:
> >def func3(self):
> >print "global var in extclass = %s" % shared.global_var
> >
> >
> > shared.py:
> > ===
> > global_var = "global"
> >
> >
> I like where this is going Walter.  I guess where I am confused is this:
>
> the globals are not static - they are set once and then won't change
> during the lifetime of the user's session.
>
> So, after messing around with the ram DB idea, I realized I was back to
> the same problem.
>
> Let's say I create a dictionary to store all of the "globals" and other
> data I want available everywhere and I put that in a class.  Every time I
> create an instance of the class to be able to access the data, a new
> instance of the dictionary is created that loads the same data in it.
>  Seems kinda inefficient for me, especially if the data changes in one
> instance - how do you keep the others all in sync, or does that just happen
> automatically?
>
> I think there is something that I fundamentally not understanding and I
> don't know what it is.   Am I making this too complicated?
>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


You are making this too complicated.

I think you are getting hung up on instantiating objects. You don't need to
instantiate an object everytime you use it. Once you have it, it's a
variable like any other and you can pass it around.

Just have a class body with your variables:

class MyObj(object):
   def __init__(self, *args, **kwargs):
   do stuff with variables here

Then instantiate the object

m = myobject(pass in your parameters)

now pass around the instantiated object on an as needed basis, or, access
the instance directly

example 1:

def func(obj):
   do stuff with obj

to call it: func(m)

example 2:

File 1: #tests

def reset_m(m, settings):
m.settings = settings

File 2: #test_two

class MyObj(object):
def __init__(self, settings=True):
self.settings = settings



if __name__ == '__main__':
m = MyObj()


File 3: #main

import test_two
import tests

m = test_two.MyObj()

print m.settings

tests.reset_m(m, False)

print m.settings

The above will print True, False

But, you don't need to create a class that has a single attribute, which is
a dictionary. In that case, you can just create a dict and pass that around
(I generally prefer to have a class body and access things using dot
notation rather than dictionary syntax)

Generally though, I think this entire idea of settings dictionary or class
can be simplified. I'll bet most of the settings have some general
similarities or are repeated, or they are the same for each user, each time.

In that case, you should probably move to a more database driven approach,
or hardcode settings in a file and override those hardcoded settings on an
as needed basis.

In django, you have a "settings" file. In there, you hard code some most of
your settings, like apps you are using, where your database resides,
middleware, etc.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Define Build Deployment

[James Bell]

I'm fairly new to software development in an enterprise environment I'm
constantly hearing the term "build deployment" and do no want to ask what
it means since it seems simple. I cannot find a definition online.

if the word is context specific please describe what it means at your
company.

thanks
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] advice on global variables

[James Reynolds]

Sent from my iPad

On Jul 10, 2012, at 3:11 PM, Chris Hare  wrote:

> 
> I know they are bad.  That is why I would prefer not to use it, but I am not 
> sure how else to handle this problem.
> 
> In this app, the user must log in.  Once authenticated, they have a userid 
> stored in the SQLite database.  Before splitting my app into multiple files, 
> I used a global variable.  I know its bad, but it worked.  Now that things 
> are split apart, the classes which used it previously now don't see it 
> anymore, even using the global keyword.  I think this is the expected 
> behavior.  See here
> 
> file: a.py
> 
> import b
> global_var = "global"
> 
> def func1():
>global global_var
>print "global var in func1 = %s" % global_var
> 
> class intclass:
>def func2(self):
>global global_var
>print "global var in intclass = %s" % global_var
> 
> print "global_var = %s" % global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
> 
> file: b.py
> 
> class extclass:
>def func3(self):
>global global_var
>print "global var in extclass = %s" % global_var
> 
> When I run it, I get what I think the expected behavior, that the external 
> class ext class won't be able to see the global_var
> 
> Big-Mac:t chare$ python a.py
> global_var = global
> global var in func1 = global
> global var in intclass = global
> Traceback (most recent call last):
>  File "a.py", line 18, in 
>g.func3()
>  File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in 
> func3
>print "global var in extclass = %s" % global_var
> NameError: global name 'global_var' is not defined
> 
> So - my question is this:  how do I solve the problem of multiple classes 
> needing to get access to a value which needs to be preserved across the 
> lifetime of the running application?
> 
> One thought was a RAM based SQLite database, but that seems like a lot of 
> work.  I dunno, maybe that is the option.
> 
> suggestions, ideas, criticisms are all welcome.  Python code aside, I just 
> don't know how to approach this problem in Python.
> 
> Thanks, as always for the feedback and guidance.
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



I'll admit it's a little tough for me to read your code.

If you need to store some information, store it to your db and access it later.

If would rather have he settings in the source code, study django's conf and 
how it handles configurations.

Better yet, why not just use django for your app?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] advice on global variables

[James Reynolds]

On Tue, Jul 10, 2012 at 4:32 PM, Prasad, Ramit wrote:

> > You should avoid using the global statement.
> >
> > In your case, I would think you could just add an argument to the method:
> >
> > class MyObj(object):
> > def __init__(self, arg):
> > self.arg = arg
> > def my_func(self, new_arg):
> > self.arg = new_arg
> >
> > to call it:
> >
> > arg = 1
> >
> > m = MyObj(arg)
> > print m.arg
> > new_arg = 2
> > m.my_func(new_arg)
> > print m.arg
>
> Just as a note, this would not really work if the variable needs to be
> changed and read from several places when the value is an immutable
> type such as numbers / strings. In that case, then you could use
> the same logic but instead place the value in a list and pass that
> and always check/update the first element of the list.
>
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


In this case, you can read the attribute of MyObj and you just pass an
instantiated MyObj around to where it is needed.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] advice on global variables

[James Reynolds]

On Tue, Jul 10, 2012 at 3:11 PM, Chris Hare  wrote:

>
> I know they are bad.  That is why I would prefer not to use it, but I am
> not sure how else to handle this problem.
>
> In this app, the user must log in.  Once authenticated, they have a userid
> stored in the SQLite database.  Before splitting my app into multiple
> files, I used a global variable.  I know its bad, but it worked.  Now that
> things are split apart, the classes which used it previously now don't see
> it anymore, even using the global keyword.  I think this is the expected
> behavior.  See here
>
> file: a.py
>
> import b
> global_var = "global"
>
> def func1():
> global global_var
> print "global var in func1 = %s" % global_var
>
> class intclass:
> def func2(self):
> global global_var
> print "global var in intclass = %s" % global_var
>
> print "global_var = %s" % global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
>
> file: b.py
>
> class extclass:
> def func3(self):
> global global_var
> print "global var in extclass = %s" % global_var
>
> When I run it, I get what I think the expected behavior, that the external
> class ext class won't be able to see the global_var
>
> Big-Mac:t chare$ python a.py
> global_var = global
> global var in func1 = global
> global var in intclass = global
> Traceback (most recent call last):
>   File "a.py", line 18, in 
> g.func3()
>   File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in
> func3
> print "global var in extclass = %s" % global_var
> NameError: global name 'global_var' is not defined
>
> So - my question is this:  how do I solve the problem of multiple classes
> needing to get access to a value which needs to be preserved across the
> lifetime of the running application?
>
> One thought was a RAM based SQLite database, but that seems like a lot of
> work.  I dunno, maybe that is the option.
>
> suggestions, ideas, criticisms are all welcome.  Python code aside, I just
> don't know how to approach this problem in Python.
>
> Thanks, as always for the feedback and guidance.
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


You should avoid using the global statement.

In your case, I would think you could just add an argument to the method:

class MyObj(object):
def __init__(self, arg):
self.arg = arg
def my_func(self, new_arg):
self.arg = new_arg

to call it:

arg = 1

m = MyObj(arg)
print m.arg
new_arg = 2
m.my_func(new_arg)
print m.arg
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Examples of "With....As Statement" in Python

[James Bell]

I'm attempting to learn how to use the "withas" statement in python.

I've read the documentation and also a few tutorials but I still cannot
understand the concept. or how this is normally used. Can someone please
write an example or 2 of simple ways to use the "with statement".

I understand in java the try...catch...finally block so maybe someone can
point out the relationship between them.

thanks.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Opening filenames with unicode characters

[James Chapman]

Thanks to everyone who responded on this thread, your time is greatly
appreciated.

It appears however that my problem is related to the environment. I
sent my original email right before leaving work and have since been
working on a physical machine without any problems. I've copied some
of that code to my remote virtual machine where I'm doing the dev work
and the same example that works on my physical win7 machine fails on
my virtual win 2008 machine. Win2008 host platform is Linux with
VirtualBox.

The only remaining question is whether this is a one off issue,
whether it's related to the virtual machine or whether it's related to
Windows 2008. I guess I'll find out tomorrow.

Oh and Tim, you'll be happy to know that regex does not affect the
string in this case. Well, at least not the way I'm using it to
extract data.

--
James

At Thursday, 28/06/2012 on 21:17 Tim Golden wrote:

On 28/06/2012 20:48, James Chapman wrote:
> The name of the file I'm trying to open comes from a UTF-16 encoded
> text file, I'm then using regex to extract the string (filename) I
> need to open.

OK. Let's focus on that. For the moment -- although it might
well be very relevant -- I'm going to ignore the regex side
of things. It's always trying to portray things like this
because there's such confusion between what characters I
write to represent the data and the data represented by those
characters themselves!

OK, let's adopt a convention whereby I represent the data as
they kind of thing you'd see in a hex editor. This obviously
isn't how it appear in a a text file but hopefully it'll be
clear what's going on.

I have a filename £10.txt -- that is the characters:

POUND SIGN
DIGIT ONE
DIGIT ZERO
FULL STOP
LATIN SMALL LETTER T
LATIN SMALL LETTER X
LATIN SMALL LETTER T

I have -- prior to your getting there -- placed this in a text
file which I guarantee is UTF16-encoded. For the purposes of
illustration I shall do that in Python code here:

with open ("filedata.dat", "wb") as f:
   f.write (u"£10.txt".encode ("utf16"))

The file is named "filedata.dat" and looks like this (per our
convention):

ff fe a3 00 31 00 30 00 2e 00 74 00 78 00 74 00

I now want to read the contents of the that file as a
filename and open the file in question. Here goes:

#
# Open the file and extract the data as a set of
# bytes into a Python (byte) string.
#
with open("filedata.dat", "rb") as f:
   data = f.read()

#
# Convert the data into a unicode object by decoding
# the UTF16 bytes
#
filename = data.decode("utf16")

# filename is now a unicode object which, depending on
# what your console offers, will either display as
# £10.txt or as \xa310.txt or as something else.

#
# Open that file by passing the unicode object directly
# to Python's file-opening mechanism
#
ten_pound_txt = open (filename, "rb")
print ten_pound_txt.read () # whatever
ten_pound_txt.close ()

I don't know if that makes anything clearer for you, but at
least it gives you something to try out.

The business with the regex clouds the issue: regex can play
a little awkwardly with Unicode, so you'd have to show some
code if you need help there.

TJG
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Opening filenames with unicode characters

[James Chapman]

Informative thanks Jerry, however I'm not out of the woods yet.

 
> Here's a couple of questions that you'll need to answer 'Yes' to
> before you're going to get this to work reliably:
> 
> Are you familiar with the differences between byte strings and unicode
> strings? 

I think so, although I'm probably missing key bits of information.

> Do you understand how to convert from one to the other,
> using a particular encoding?  

No not really. This is something that's still very new to me.

> Do you know what encoding your source
> file is saved in? 

The name of the file I'm trying to open comes from a UTF-16 encoded text file, 
I'm then using regex to extract the string (filename) I need to open. However, 
all the examples I've been using here are just typed into the python console, 
meaning string source at this stage is largely irrelevant.

> If your string is not coming from a source file,
> but some other source of bytes, do you know what encoding those bytes
> are using?
> 
> Try the following.  Before trying to convert filename to unicode, do a
> "print repr(filename)".  That will show you the byte string, along
> with the numeric codes for the non-ascii parts.  Then convert those
> bytes to a unicode object using the appropriate encoding.  If the
> bytes are utf-8, then you'd do something like this:
> unicode_filename = unicode(filename, 'utf-8')

>>> print(repr(filename))
"This is_a-test'FILE to Ensure$ that\x9c stuff^ works.txt.js"

>>> fileName = unicode(filename, 'utf-8')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 35: invalid 
start byte

>>> fileName = unicode(filename, 'utf-16')

>>> fileName
u'\u6854\u7369\u6920\u5f73\u2d61\u6574\u7473\u4627\u4c49\u2045\u6f74\u4520\u736e\u7275\u2465\u7420\u6168\u9c74\u7320\u7574\u\u205e\u6f77\u6b72\u2e73\u7874\u2e74\u736a'



So I now have a UTF-16 encoded string, but I still can't open it.

>>> codecs.open(fileName, 'r', 'utf-16')
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\codecs.py", line 881, in open
file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: 
u'\u6854\u7369\u6920\u5f73\u2d61\u6574\u7473\u4627\u4c49\u2045\u6f74\u4520\u736e\u72
75\u2465\u7420\u6168\u9c74\u7320\u7574\u\u205e\u6f77\u6b72\u2e73\u7874\u2e74\u736a'


I presume I need to perform some kind of decode operation on it to open the 
file but then am I not basically going back to my starting point?

Apologies if I'm missing the obvious.

--
James


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Opening filenames with unicode characters

[James Chapman]

Why can I not convert my existing byte string into a unicode string?

In the mean time I'll create my original string as unicode and see if
that solves my problem.

>>> fileName = unicode(filename)
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position
35: invalid start byte

--
James

At Thursday, 28/06/2012 on 18:58 Jerry Hill wrote:

On Thu, Jun 28, 2012 at 1:55 PM, James Chapman  wrote:
> Thanks Tim, while this works, I need the name to be stored in a
variable as
> it's dynamic.
>
> In other words, how do I rewrite
> open(u"blah£.txt")
>
> to be
> filename = "blah£.txt"
> open(filename)

You have to create a unicode-string, not a byte-string.  So, like
this:

filename = u"blah£.txt"
open(filename)

-- 
Jerry


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Opening filenames with unicode characters

[James Chapman]

Thanks Tim, while this works, I need the name to be stored in a
variable as it's dynamic.

In other words, how do I rewrite 
open(u"blah£.txt")

to be 
filename = "blah£.txt"
open(filename)

At Thursday, 28/06/2012 on 18:39 Tim Golden wrote:

On 28/06/2012 18:19, James Chapman wrote:
> Hi there python list.
>
> I'm trying to open a text file named "This is_a-test'FILE to Ensure$
> that£ stuff^ works.txt" (without the quotes) but I'm struggling to
> find a way to open it.

Happily, you're using Windows, which makes this very much easier.

Short Explanation

Do this: open (u"blah £3.50.txt").read () # note the u- prefix

Long Explanation:

There's way too many levels of indirection between you and the
disk to go into it all, but basically you're telling Python to
create a (byte) string from those characters using its default
encoding for code, which is UTF-8. UTF-8 maps "£" to the
one-byte 0x9c. Somewhere in the layers between that string and
the representation on disk, a mismatch occurs.

TJG
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Opening filenames with unicode characters

[James Chapman]

Hi there python list.

I'm trying to open a text file named "This is_a-test'FILE to Ensure$
that£ stuff^ works.txt" (without the quotes) but I'm struggling to
find a way to open it.

>>> filename = "This is_a-test'FILE to Ensure$ that£ stuff^
works.txt.js"
>>> open(filename)
Traceback (most recent call last):
  File "", line 1, in 
IOError: [Errno 2] No such file or directory: "This is_a-test'FILE to
Ensure$ that\x9c stuff^ works.txt.js"

>>> import codecs
>>> codecs.open(filename, 'r', 'utf-8')
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\codecs.py", line 881, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: "This is_a-test'FILE to
Ensure$ that\x9c stuff^ works.txt.js"

>>> os.listdir(".")
["This is_a-test'FILE to Ensure$ that\xa3 stuff^ works.txt.js"]

>>> filename.decode()
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position
35: ordinal not in range(128)

I've tried all sorts of encode and decode methods on the string
containing the file name but nothing seems to be working. 

Any help would be appreciated.

James

PS: This is on 64bit Windows with ActivePython 2.7.3


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] web design and python

[James Reynolds]

On Thu, Jun 21, 2012 at 6:53 PM, Tamar Osher wrote:

>  I have many questions, and eagerly ask you to please respond to me.  I
> value your expertise, and greatly appreciate you taking the time to share.
>  I want to find out, in elaborate detail, about the Python/Django
> relationship to web design.  Why does a person or a company seek out and
> choose a Python-Django website, instead of Drupal, or Joomla, or Wordpress,
> or something else?  What is the description of the unique, small niche that
> Python/Django has in the world of web design?  Who are these individuals
> and companies that want Python/Django/TurboGears websites?  What is the
> Python/Django/TurboGear relationship to "responsive web design" (which
> means the website is beautifully, perfectly view-able on all sizes of
> screens, including a smartphone, an ipad, and a desktop). What are the pros
> and cons of choosing Python/Django/TurboGears, compared to other web
> development options, such as Drupal, Joomla, and Wordpress?  Is it really
> true that Python/Django websites cannot use just any web hosting service,
> and are more expensive to maintain?  I have read several books about Python
> web development, and have extensively researched online, but have not found
> any of this information.  It would be nice if someone could write a full
> article about this, to explain the true, current situation and realistic
> 2012 options to beginners who don't have degrees in computer science.  I am
> very, very eager to hear from each one of you.  THANK YOU for your time and
> help!
>
>
> **
> *
>
>
>
> *
>
> * *
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


I'm not really sure how to respond to you, but I'll give it a go.

 Why does a person or a company seek out and choose a Python-Django
> website, instead of Drupal, or Joomla, or Wordpress, or something else


Well, Drupal, Joomla, Wordpress are all CMS (content management Systems).
They are also written in PHP. I think the goals of developing in django vs
having a wordpress site are quite a bit different, although they could be
the same, I suppose.

For example, if you wanted a site to just publish your daily thoughts, you
might use wordpress. However, if you want to identify a tree by an image
you took on your iphone, you might use django (and a lot of other stuff to
make that magic happen!)

Generally, I think, companies and people choose django because of the
community and how much you can get done with it in a short period of time.

What is the description of the unique, small niche that Python/Django has
> in the world of web design?


Not sure what this sentence means, but I wouldn't call Python / Django as
small niche in the world of web design. It's probably the third most
popular option right now, behind PHP and rails.

  What is the Python/Django/TurboGear relationship to "responsive web
> design" (which means the website is beautifully, perfectly view-able on all
> sizes of screens, including a smartphone, an ipad, and a desktop)


Turbogears is another framework.  I'm not sure of many sites built on it;
I'm sure they are out there, I just don't know of them.

You should check out: http://www.djangosites.org/

  Is it really true that Python/Django websites cannot use just any web
> hosting service, and are more expensive to maintain?


Django is just a set of libraries written in python. So long as the
webhosting service allows you to install python, install apache (or
something similar) and allows you to broadcast, you can host the website.
As for the second part, more expensive than what? Once you get the site,
they sort of run themselves. You just have to pay for bandwidth and
hosting. If your saving a lot of data and using a lot of bandwidth, that
could get costly, but... that is not an issue of django, but rather of
design / product needs.

Some webhosting companies make it easier to set up django though. Some
already have it installed, some do not.

to beginners who don't have degrees in computer science


I also do not have a degree in computer science. I am also a full time web
developer working in Python / Django.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Pymongo Error

[James Reynolds]

On Tue, Jun 19, 2012 at 1:22 AM, Ranjith Kumar wrote:

> Hi all,
> I tried Django with Mongodb while running manage.py syncdb I endup with
> this error
>
> note : it works fine with sqlite and mysql db
>
> (django-1.3)ranjith@ranjith:~/
> sandbox/python-box/hukkster-core-site/hukk$ ./manage.py syncdb
> /home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/pymongo/connection.py:385:
> UserWarning: must provide a username and password to authenticate to
> hukkster_testing
>   "to authenticate to %s" % (db,))
> Creating tables ...
> Traceback (most recent call last):
>   File "./manage.py", line 14, in 
> execute_manager(settings)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/__init__.py",
> line 438, in execute_manager
> utility.execute()
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/__init__.py",
> line 379, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/base.py",
> line 191, in run_from_argv
> self.execute(*args, **options.__dict__)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/base.py",
> line 220, in execute
> output = self.handle(*args, **options)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/base.py",
> line 351, in handle
> return self.handle_noargs(**options)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py",
> line 109, in handle_noargs
> emit_post_sync_signal(created_models, verbosity, interactive, db)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/core/management/sql.py",
> line 190, in emit_post_sync_signal
> interactive=interactive, db=db)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py",
> line 172, in send
> response = receiver(signal=self, sender=sender, **named)
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py",
> line 41, in create_permissions
> "content_type", "codename"
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/db/models/query.py",
> line 107, in _result_iter
> self._fill_cache()
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/db/models/query.py",
> line 772, in _fill_cache
> self._result_cache.append(self._iter.next())
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/django/db/models/query.py",
> line 959, in iterator
> for row in self.query.get_compiler(self.db).results_iter():
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/djangotoolbox/db/basecompiler.py",
> line 229, in results_iter
> for entity in self.build_query(fields).fetch(low_mark, high_mark):
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/djangotoolbox/db/basecompiler.py",
> line 290, in build_query
> query.order_by(self._get_ordering())
>   File
> "/home/ranjith/virtualenvs/django-1.3/local/lib/python2.7/site-packages/djangotoolbox/db/basecompiler.py",
> line 339, in _get_ordering
> raise DatabaseError("Ordering can't span tables on non-relational
> backends (%s)" % order)
> django.db.utils.DatabaseError: Ordering can't span tables on
> non-relational backends (content_type__app_label)
>
>
> DB settings in settings.py
>
> DATABASES = {
> 'default': {
> 'ENGINE': 'django_mongodb_engine',
> 'NAME': 'helloworld',
> 'USER': '',
> 'PASSWORD': '12424214',
> 'HOST': 'mongodb://staff.mongohq.com/',
> 'PORT': 'X',
> },
> }
>
> my requirement packages,
> Django==1.3
> dictshield==0.4.4
> django-mongodb-engine==0.4.0
> django-social-auth==0.6.9
> djangotoolbox==0.0.1
> httplib2==0.7.4
> mongoengine==0.6.10
> mongokit==0.8
> oauth2==1.5.211
> pymongo==2.2
> python-openid==2.2.5
> simplejson==2.5.2
> wsgiref==0.1.2
>
> Please point me what i missed here...
>
> --
> Cheers,
> Ranjith Kumar K,
> Chennai.
>
> http://ranjithtenz.wordpress.com
>
>
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>


This list here is for new users to python. Using mongo with django is a
pretty advanced topic. I would suggest posting this on stackoverflow, or to
the django-users group (and there are a ton of resources on this already on
those two sites on this topic)

That said, having used mongo with django, I'll bet your problem is that you
are using the django version provided by the django pro

Re: [Tutor] Dictionary

[James Reynolds]

Does this language have grammar independent of english?

If no, just use .split() on the string and loop through that.

If yes, well, its much more complicated
On Jun 17, 2012 2:27 PM, "Selby Rowley-Cannon" 
wrote:

> Version: 2.7
> OS: Ubuntu 12.04 LTS
>
> I am writing a small translation app for Rydish (A language that exists in
> the same way Klingon does, invented by my company for a[n] RPG).
> Here is my current method of translation:
>
> Edictionary = {'English keys':'Rydish values'}
> TextEng = raw_input('Please enter your text: ')
> if TextEng in Edictionary:
> print(TextEng + ' traslates to ' + Edictionary[TextEng])
>
> But I have found that this is only going to translate one word at a time.
> I thought about a loop of somesort, but I can't seem to find one that won't
> still force the user to translate one word at a time. Can anyone tell me
> how to translate a full sentance/several sentances?
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Web Questionnaire

[James Reynolds]

I would say start using django.

The best way to learn is by doing.

This will let you learn about databases, servers, webapps, and you can get
some functionality out of it. Just start going through the django tutorial.
On Jun 4, 2012 11:57 AM, "Joel Goldstick"  wrote:

> On Mon, Jun 4, 2012 at 11:12 AM, Akeria Timothy 
> wrote:
> > Hello All,
> >
> > I am still new to learning Python and I wanted to know if anyone knew of
> a
> > script out there that would allow users to answer three basic questions,
> hit
> > submit and send that information to an excel spreadsheet?
> >
> > So, I would want them to submit the following information:
> >
> > Name:
> > Dept:
> > Software needed:
> > Semester requesting software for:
> >
> > I work for a college and I wanted to streamline the software requests for
> > the school year.
> >
> > Is this a basic script? I have started my script but I know it will
> probably
> > take me longer to write it so I figured I would ask if there was
> something
> > out there that I could just modify.
> >
> >
> > Thanks in advance
> >
> > ___
> > Tutor maillist  -  Tutor@python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
> >
> I'm not sure this is a 'python tutor' question.  You would need to
> have a website that presumable limits access to certain people so that
> anyone couldn't make a request.  Django could do that for you out of
> the box more or less, but then you would have to deploy it. So, its
> not just a small python script, but a website you would need to build.
>  Are you up to that yet?
>
> If you are thinking up useful projects to expand your python skills,
> perhaps publishing an email box for requests would be easier.  You
> could read the emails and enter the data in a simple command line
> program you could write in python to added data in csv format to a
> file.  This file could be viewed in excel.
>
> Such a project would help you learn about csv module, and how to
> request data from a user. -- both useful things to learn
>
>
> --
> Joel Goldstick
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Connecting to MySQLdb

[James Reynolds]

Virtualenv works just fine in windows
 On Jun 1, 2012 7:20 PM, "Corey Richardson"  wrote:

> On Fri, 1 Jun 2012 19:01:00 -0400
> Nicholas Picciano   wrote:
>
> > Hello,
> >
> > I have downloaded MySQLdb 1.2.3 from:
> >
> > http://pypi.python.org/pypi/MySQL-python
> >
> > Also, I'm using Python 2.7, but I can't even get past here:
> >
> > >>> import MySQLdb
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > ImportError: No module named MySQLdb
> >
> > Thanks
>
> Did you install it?
>
> Assuming UNIX(-like):
>
> I highly, *highly* recommend you use a virtualenv [1] and, if you want
> a fancy UI, virtualenvwrapper [2]. From there you can install all the
> packages you want with pip, without having to worry about mucking up
> the system, versions of things, etc.
>
> Assuming Windows:
>
> Good luck!
>
> --
> Corey Richardson
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class

[James Reynolds]

On Fri, Jun 1, 2012 at 9:12 AM, Adam  wrote:

> I'm working on a class that handles multiple rooms that generate a large
> amount of data. Currently my class model looks something like this (more
> apologies for any indentation errors):
> Class Model:
>rooms= {}
>for z in range(num_of_zones):
>for i in range(24):
>tmp[i] = { VAR:0, SD:0, AVG:0, READINGS:[] }
>tmp[i]['updated'] = datetime.utcnow()
>for j in OT_RANGE:
>tmp[i][j] = { VAR:0, SD:0, AVG:0, READINGS:[] }
>rooms[z] = tmp
>
> In case that gets complicated, I'm looking to store readings based off the
> current hour and current outside temperature. The Model class has other
> variables and functions that it uses to compare and manipulate the data as
> it comes in.
>
> My question is in regards to the storing all this data in a dictionary,
> which makes it easy to reference self.rooms[z][12][85][AVG]; however is
> this better/worse or the same as creating a 'Room' class to store the data?
> Since a 'Room' only contains a large amount of data, but doesn't contain
> any functions, which form of storing the data is considered 'better'?
>
> Thank you,
> Adam
> __**_
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor
>

If you have functions (or in this case methods) which will be used on a
certain 'objects' data structure, a class representation of the data is the
natural fit. Otherwise, you will need to pass the dictionary around to
various functions and this will be seriously confusing.

However, you probably should consider how to represent that data within a
class body, that is, it is extremely confusing to access data such
as self.rooms[z][12][85][AVG].

Instead, you should break down what the data structure into like-things and
how they are related, for example, you might consider something like this:

class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp = temp
self. temp = precipitation
self. temp = humidity
self. temp = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here

However, if you need to either further define the datapoints of if you want
the datapoints to be able to "do things" then you could do something like
this:

class Temperature(object):
def __init__(self, temp):
self. temp = temp
self.other_stuff_about_temp = stuff

def my_temp_func(self, *args,**kwargs):
pass #do stuff here


class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp =  Temperature(temp)
self.precipitation = precipitation
self.humidity = humidity
self.bara_pres = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here

from here you could either put your instantiated objects into a list and do
math on the elements thereof, or you could put them into another class
object like this:

class WeatherOverTime(object):
def __init__(self, my_list):
self.my_list =  my_list

def avg(self, attr): #attr is the attribute you want to take an average
of, so humidity, temp, whatever.
temp_list = map(lambda x: getattr(x, attr), my_list)
return sum(temp_list) / count(temp_list)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How install Pyramid on a Windows 7 computer?

[James Reynolds]

I don't think django 1.5 is ready, but they are targeting python3k. There
is a fork of django i think on bitbucket that works as well.
On May 31, 2012 12:33 PM, "Vince Spicer"  wrote:

> Step 1)  install Ubuntu
>
> OK sorry couldn't resist.
>
> This should help.
>
> http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/install.html#installing-pyramid-on-a-windows-system
>
> Vince
>
> On Thu, May 31, 2012 at 10:20 AM, Tamar Osher 
> wrote:
>
>>  Months ago, I learned Python version 3.  I have read a few books, and
>> want to learn how to use Python for the web.  Pyramid is a Python web
>> framework that is ready for Python version 3, but I don't know how to
>> install it on my Windows 7 computer.  Online, there are no Pyramid
>> installation instructions that discuss how to install Pyramid on a Windows
>> 7 computer.  Can someone please help me?
>>
>> from Tamar Osher, emeraldoff...@hotmail.com
>>
>>
>> **
>> *
>>
>>
>>
>> *
>>
>> * *
>>
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Thanks Everyone!!!

[James Stauble]

Thanks for the tips everyone! I am fairly new to programming and am finding
myself both bewildered and amazed. Its fun when it works, but boy oh boy,
when it doesn't
Anyway thanks again, you all have been very helpful. [?]
<<330.gif>>___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor