Re: [Tutor] Python Imported Code

[Albert-Jan Roskam]

On 25 Jun 2019 15:50, stephen.m.sm...@comcast.net wrote:

Introduction:

I have written a 'program' that does some reasonable screen scraping off of
a specific website. The program has gotten too large so I have tried to
segment it into logical pieces (tkinter logic as a start) but I am having
problems. Specifically I need to pass several dictionaries to the module
(imported code) that validates some user selection and into the code that
navigates through the website.

》》 Hi, not sure if you could use it here, but I was triggered by the term 
"validation":
* Tkinter allows you to define a validationcommand, 
https://stackoverflow.com/questions/4140437/interactively-validating-entry-widget-content-in-tkinter
* You can also define tracers. These are functions that are triggered when a 
Tkinter var (e.g  StringVar) is changed.
* You can also bind events to a function. For example a  event from 
an Entry checks the entry contents and colors it red if it's invalid.

It's been a while since I've used this but those tricks may come in handy!

Albert-Jan

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How would I replace the data of a Pandas Series with the values of a dictionary?

[Albert-Jan Roskam]

On 16 Jul 2019 23:31, Daniel Bosah  wrote:

Hi all,

I have a problem trying to match items in a dict and pandas series in
Python.

I have a dict ( called city_dict )of cities and city_id's ; for each city (
which is a key in the dict ), a unique city_id is a value in that dict.

So for example, city_dict = { New York : 1001, LA : 1002, Chicago : 1003 }.
New York is a key, 1001 is a value.

Now I have a panda Series called dfCities. In this series is a bunch of
cities, including the cities in city_dict.

My goal is to replace the cities in dfCities with the city_id's in a brand
new csv file. So if dfCities has New York in it, I want to replace it with
it's value in the dictionary, so 1001.


=》 check out 
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.replace.html.
 It accepts a dict, see also examples below.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Fwd: Re: Unexpected result when running flask application.

[Albert-Jan Roskam]

Ooops, forgot to 'reply all'
-- Forwarded message --
From: Albert-Jan Roskam 
Date: 28 Jun 2019 21:31
Subject: Re: [Tutor] Unexpected result when running flask application.
To: Cameron Simpson 
Cc:



On 20 Jun 2019 00:56, Cameron Simpson  wrote:

On 19Jun2019 09:54, Alan Gauld  wrote:
>On 19/06/2019 05:18, Cravan wrote:
>> I am experiencing an unexpected result when I try to
>> run my flask application.
>> The movie.html page prints out nothing except those in the . This 
>> appears on my webpage:
>
>Note that the mail server does not allow (for security reasons)
>binary attachments so we lost your image.

Cravan, you might find it useful to "View Source" of that page in your
browser.

You can also use command line tools like "curl" or "wget" to directly
fetch the page content.

>However, your html files are not in HTML.
>I'm not a Flask expert but every time I've used Flask the
>html pages have been real HTML. Yours appear to be in some
>strange pseudo markup language.

It is very common in Flask to write HTML pages using Jinja templates,
which is what his examples look like.

Of course this adds more complexity, if he forgets to use Jinja to
render the content to HTML before returning it.

>If this is something unique to Flask then I suspect you will
>need to ask on a Flask support page or list. It doesn't seem
>to be a Python language related issue at this point.

==》 I haven't seen the templates, but your view function should probably retun 
flas.render_template('some.html', **kwargs), where kwargs will be e.g. your 
database values that are going to be displayed/{{interpolated}}.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interoperating with Excel, was Re: Questions

[Albert-Jan Roskam]

On 7 Apr 2019 14:07, Peter Otten <__pete...@web.de> wrote:

Diana Katz wrote:

> 1) Can you use python from excel? Or just export to excel?
> 2) I am trying to see if there's a way using python to automate all of
> this work that I need to do. I have to collect quarterly segment data for
> hundreds of public companies and go back at least 12-16 quarters. We use
> an aggregator like factset and they actually don't have this option
> available in an automated way. So I'm trying to see if there's a way to
> build this. Basically, I get my data from sec.gov and they have
> interactive data - they even have the data in excel (though it's a messy
> file and hard to read). I attached some of the steps and the data that i'd
> want to see. Basically i'd want the excel to look like:
> old to new quarters - going back 12 to 16 quarters (more if possible but
> not if it will stop the project).
>  Columns: 3/31/2017, 6/30/2017, 9/30/17, 12/31/17, 3/313/2018...
> Rows:
> Sales for segment A
> Sales for Segment b
> Sales for SEgment C
> …(for as many segments as they have)
>
> Earnings for Segment A
> .Earnings for Segment B
>
> Depreciation for Segment A
> Depreciation for Segment B
> Depreciation for Segment C...

These look like "pivot tables" which are well supported by Excel.
I expect that this is easy to automate with a little bit of Basic.

Of course you can build these tables with a Python script if you feel more
comfortable in Python. Then either write them into csv files ("comma
separated value", supported by the standard library)

https://docs.python.org/3/library/csv.html

which can be read by Excel -- or use a dedicated library. Google came up
with

https://xlsxwriter.readthedocs.io/


===>> There's xlrd for xls files, openpyxl for xlsx files. And pandas can also 
read excel, and you can use groupby, pivottable etc. there:
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_excel.html.
 With the win32com library you can also automate Excel (Windows only).

Excel is evil, especially if the files are made by humans and come from 
different sources.




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] is there a graphics library for common tkinter Button functions?

[Albert-Jan Roskam]

On 21 Mar 2019 12:11, Alan Gauld via Tutor  wrote:

On 21/03/19 00:28, Chris Roy-Smith wrote:

> Yes I knew that buttons need a function to do anything.
>
> I was hoping that the wheel didn't need re-inventing.

===>> A few are available: 
https://anzeljg.github.io/rin2/book2/2405/docs/tkinter/bitmaps.html. But you 
could use Material Design: https://material.io/tools/icons/?style=baseline



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question for tutoring page

[Albert-Jan Roskam]

On 13 Mar 2019 18:14, Alan Gauld via Tutor  wrote:

On 11/03/2019 16:10, Diana Katz wrote:
> What is the best way to ..program using python - that could recognize
> a 3D object and then rank drawings of the object as to which are more
> accurate.

===>> check this out: 
https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] schedulers

[Albert-Jan Roskam]

On 28 Feb 2019 15:45, nathan tech  wrote:

Hi there,

I recently started working on a backup program, and the one big feature
everyone wants in backup programs is the ability to schedule backups, right?

but I'm thinking, should I do this?


==》 this might interest you: https://docs.python.org/3/library/sched.html. Or 
simply the Windows Task Scheduler?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Albert-Jan Roskam]

On 8 Feb 2019 19:18, Chip Wachob  wrote:

Hello,

I've been off working on other projects, but I'm finally back to the
project that so many here have helped me work through.  Thank you to the
group at large.

So, this leads me to my question for today.

I'm not sure what the "correct" term is for this, but I want to create what
I'll call a Package.

I want to bundle all my scripts, extra libraries, etc into one file.  If I
can include a copy of Python with it that would be even better.


==》 Hi, also check out pyscaffold or the similar cookiecutter to generate 
"package skeletons". Directory structure, default setup.py, Readme.md file 
template, etc etc. I've never used them, but py2exe or cx_freeze might also 
interest you.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Albert-Jan Roskam]

On 10 Nov 2018 01:03, Steven D'Aprano  wrote:

On Thu, Nov 08, 2018 at 11:34:35AM -0500, Avi Gross wrote:
> An interesting discussion that is outside the scope of a group like this is
> HOW malicious things can be done and perhaps how to avoid them.
>

Isn't the rule, simply:

this_is_stupid = eval(input("please enter malicious code: "))

... and other uses range from 'code smell' to 'elegant' (where namedtuple is an 
example of the latter)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Albert-Jan Roskam]

On 8 Nov 2018 17:34, Avi Gross  wrote:

> What can you do to minimize risks in such > situations?

ast.literal_eval, with a smallish maximum string length? 
https://docs.python.org/3/library/ast.html.

(That's exactly the only ast function that know! :-)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regex for Filesystem path (Asad)

[Albert-Jan Roskam]

From: Tutor  on behalf of 
Alan Gauld via Tutor 
Sent: Thursday, November 8, 2018 10:49 AM
To: tutor@python.org
Subject: Re: [Tutor] Regex for Filesystem path (Asad) 
 


On 08/11/2018 02:55, Asad wrote:

 > Why is it putting \  this breaks the unix path it should be:
 > 
 > /a/b/c/d/test/28163133/22326541   ===> for unix platform logs
 > 
 > \a\b\c\d\test\28163133\22326541  ===> for windows platform logs

Hi,

Maybe pathlib will be of use here:
>>> from pathlib import Path
>>> Path('c:\\beeh/foo\\bar/baz').as_posix()
'c:/beeh/foo/bar/baz'

I was thinking it would also be possible to do (in Windows):
import os.path
os.path.sep = '/'
os.path.normpath('c:\\beeh/foo\\bar/baz')

But alas, this still creates normalized windows-style paths.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Albert-Jan Roskam]

On 7 Nov 2018 20:36, Mats Wichmann  wrote:
Not sure what you're after, but what's wrong with just setting them?

Parent.avar = "a class var"

Hi Alan, Mats,

I should have mentioned that the code is part of a function sql_to_hdf5. So it 
should be able to convert an *arbitrary* Sql server table into hdf5 format*). 
But even if it was just one specific table: it's quite cumbersome to write the 
column names (class variable names) and the target hdf5 data types (class 
variable values) for, say, a hundred columns. This is an example from the 
pytables website:


>>> from tables import *
>>> class Particle(IsDescription):
... name  = StringCol(16)   # 16-character String
... idnumber  = Int64Col()  # Signed 64-bit integer
... ADCcount  = UInt16Col() # Unsigned short integer
... TDCcount  = UInt8Col()  # unsigned byte
... grid_i= Int32Col()  # 32-bit integer
... grid_j= Int32Col()  # 32-bit integer
... pressure  = Float32Col()# float  (single-precision)
... energy= Float64Col()# double (double-precision)


Imagine having to write this for 100 columns, brrr.

So the code grabs the sql column names and the datatypes (easy with sqlalchemy) 
and translates that into the hdf5 equivalent. And pytables uses a class, with 
class variables, to define this.

A classmethod might be nice. I've never used this before, but I'll try this (I 
did try __new__ this afternoon, but it looked even more complicated). And it 
just occurred to me that exec() might even be a way (namedtuple does it, too). 
Or should I now grab my coat for even mentioning exec? :-)))

Best wishes,
Albert-Jan

*) I tried using pandas to_hdf for this, but this does not work wel with mixed 
dtypes, NULL values, and chunkwise reading.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] best way to dynamically set class variables?

[Albert-Jan Roskam]

Hi,

Background: In my code I use sqlalchemy to read SQL server data. I want to 
write that data to HDF5 using pytables (see 'declaring a column descriptor': 
https://www.pytables.org/usersguide/tutorials.html). My question is not about 
pytables or sqlalchemy per se, but I thought it would be informative to mention 
this. 

What is the best way to dynamically set class variables? I am looking for a 
generalization of something like this:

class Parent: pass
class Child(Parent):
col1 = 'str'
col2 = 'int'

Several (im)possible solutions:

# ---
class Parent: pass
class_vars = dict(col1='str', col2='int')

# approach 1
Child = type('Child', (Parent,), class_vars)

# approach 2
class Child(Parent): pass
Child.__dict__.update( class_vars )  # AttributeError: 'mappingproxy' object 
has no attribute 'update'

# approach 3
class Child(Parent): pass
for k, v in class_vars.items():
setattr(Child, k, v)

I initially chose approach #1, but I find this way of defining a class quite 
cryptic (but then, it's part of the language definition!). What's the best way 
to do this? I am using Python 3.5 (Windows). Thanks in advance!

Best wishes,
Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Albert-Jan Roskam]

On 19 Oct 2018 18:09, Pat Martin  wrote:

TLDR; How do you figure out if code is inefficient (if it isn't necessarily
obvious) and how do you find a more efficient

-
Hi, check this: https://docs.python.org/3/library/profile.html. Also, the 
timeit module is handy for smaller snippets.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] what does the forward slash mean in this function signature?

[Albert-Jan Roskam]

From: eryk sun 
Sent: Monday, October 15, 2018 12:10 PM
To: tutor@python.org
Cc: sjeik_ap...@hotmail.com
Subject: Re: [Tutor] what does the forward slash mean in this function 
signature?
  

On Mon, Oct 15, 2018 at 6:00 AM Albert-Jan Roskam
 wrote:
>
> In Python 3.6 (Windows) I often see a forward slash in a function signature, 
> see below.
> What does it mean? I vaguely remember reading something about new 
> possbilities in
> python 3, something like "def foo(x, *, y)". Perhaps it's related to that?
>
> >>> help({}.fromkeys)
> Help on built-in function fromkeys:
>
> fromkeys(iterable, value=None, /) method of builtins.type instance
> Returns a new dict with keys from iterable and values equal to value.

This syntax is for position-only function parameters. It's used by
Argument Clinic [1], a preprocessor used internally to develop
CPython. Position-only parameters have also been proposed for the
language grammar, first in PEP 457 and again in PEP 570 [2].

[1]:  https://docs.python.org/3/howto/clinic.html#converting-your-first-function
[2]: https://www.python.org/dev/peps/pep-0570



> Hi Eryk,

Thank you, that's interesting to know. I wasn't even aware of position-only 
parameters until I read about them. Of course I've used ord() many times, I 
just thought it has a keyword argument. Why even bother to have positional-only 
params? I agree that ord('a') reads more easily than ord(character='a'), but 
other than that I don't see an advantage to enforce it.

The pages you mentioned also made me read about function annotations and type 
hints. Do you know why type hints were designed while there are already are 
parseable docstring notations (sphinx  :param foo: int, numpydoc (much prettier 
IMHO), and others)? The type hints remind of Cython, but "Type annotations 
should not be confused with variable declarations in statically typed 
languages. The goal of annotation syntax is to provide an easy way to specify 
structured type metadata for third party tools." [1]

Thanks!

Albert-Jan

[1]  https://www.python.org/dev/peps/pep-0526/
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] what does the forward slash mean in this function signature?

[Albert-Jan Roskam]

Hello,


In Python 3.6 (Windows) I often see a forward slash in a function signature, 
see below. What does it mean? I vaguely remember reading something about new 
possbilities in python 3, something like "def foo(x, *, y)". Perhaps it's 
related to that?


>>> help({}.fromkeys)
Help on built-in function fromkeys:

fromkeys(iterable, value=None, /) method of builtins.type instance
Returns a new dict with keys from iterable and values equal to value.


Thanks!


Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] need help generating table of contents

[Albert-Jan Roskam]

From: Tutor  on behalf of 
Peter Otten <__pete...@web.de>
Sent: Monday, August 27, 2018 6:43 PM
To: tutor@python.org
Subject: Re: [Tutor] need help generating table of contents
  

Albert-Jan Roskam wrote:

> 
> From: Tutor  on behalf
> of Peter Otten <__pete...@web.de> Sent: Friday, August 24, 2018 3:55 PM
> To: tutor@python.org
> 
>> The following reshuffle of your code seems to work:
>> 
>> print('\r\n** Table of contents\r\n')
>> pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'
>> 
>> def process(triples, limit=None, indent=0):
>> for index, (title, page, count) in enumerate(triples, 1):
>> title = indent * 4 * ' ' + title
>> print(title.ljust(79, ".") + page.zfill(2))
>> if count:
>> process(triples, limit=int(count), indent=indent+1)
>> if limit is not None and limit == index:
>>  break
>> 
>> process(iter(re.findall(pattern, toc, re.DOTALL)))
> 
> Hi Peter, Cameron,
> 
> Thanks for your replies! The code above indeeed works as intended, but: I
> don't really understand *why*. I would assign a name to the following line
> "if limit is not None and limit == index", what would be the most
> descriptive name? I often use "is_*" names for boolean variables. Would
> "is_deepest_nesting_level" be a good name?



> No, it's not necessarily the deepest level. Every subsection eventually ends 
> at this point; so you might call it reached_end_of_current_section
> 
> Or just 'limit' ;) 

LOL. Ok, now I get it :-)

> The None is only there for the outermost level where no /Count is provided. 
> In this case the loop is exhausted.
> 
> If you find it is easier to understand you can calculate the outer count aka 
> limit as the number of matches - sum of counts:
> 



>> Also, I don't understand why iter() is required here, and why finditer()
> >is not an alternative.

>finditer() would actually work -- I didn't use it because I wanted to make 
> as few changes as possible to your code. What does not work is a list like 
>the result of findall(). This is because the inner for loops (i. e. the ones 
>in the nested calls of process) are supposed to continue the iteration 
>instead of restarting it. A simple example to illustrate the difference:

Ah, the triples cannot be unpacked inside the "for" line of the loop. This 
works:
def process(triples, limit=None, indent=0):
 for index, triple in enumerate(triples, 1):
 title, page, count = triple.groups()  # unpack it here
 title = indent * 4 * ' ' + title
 print(title.ljust(79, ".") + page.zfill(2))
 if count:
 process(triples, limit=int(count), indent=indent+1)
 if limit is not None and limit == index:
 break

process(re.finditer(pattern, toc, re.DOTALL))


If I don't do this, I get this error:
  File "Q:/toc/toc.py", line 64, in 
process(re.finditer(pattern, toc, re.DOTALL))
  File "Q:/Ctoc/toc.py", line 56, in process
for index, (title, page, count) in enumerate(triples, 1):
TypeError: '_sre.SRE_Match' object is not iterable

Process finished with exit code 1


Thanks again Peter! Very insightful!

Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] need help generating table of contents

[Albert-Jan Roskam]

From: Tutor  on behalf of 
Peter Otten <__pete...@web.de>
Sent: Friday, August 24, 2018 3:55 PM
To: tutor@python.org

> The following reshuffle of your code seems to work:
> 
> print('\r\n** Table of contents\r\n')
> pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'
> 
> def process(triples, limit=None, indent=0):
>     for index, (title, page, count) in enumerate(triples, 1):
>     title = indent * 4 * ' ' + title
>     print(title.ljust(79, ".") + page.zfill(2))
>     if count:
>     process(triples, limit=int(count), indent=indent+1)
>     if limit is not None and limit == index:
>     break
> 
> process(iter(re.findall(pattern, toc, re.DOTALL)))

Hi Peter, Cameron,

Thanks for your replies! The code above indeeed works as intended, but: I don't 
really understand *why*.
I would assign a name to the following line "if limit is not None and limit == 
index", what would be the most descriptive name? I often use "is_*" names for 
boolean variables. Would "is_deepest_nesting_level" be a good name?

Also, I don't understand why iter() is required here, and why finditer() is not 
an alternative.

I wrote the bookmarks file myself, and the code above is part of a shell script 
that compiles a large .pdf, with openoffice commandline calls, ghostscript, 
git, pdftk and python. The human-readable toc and the pdf bookmarks will always 
be consistent if I only need to edit one file.

Thanks again!

Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] need help generating table of contents

[Albert-Jan Roskam]

Hello,

I have Ghostscript files with a table of contents (toc) and I would like to use 
this info to generate a human-readable toc. The problem is: I can't get the 
(nested) hierarchy right.

import re

toc = """\
[ /PageMode /UseOutlines
  /Page 1
  /View [/XYZ null null 0]
  /DOCVIEW pdfmark
[ /Title (Title page)
  /Page 1
  /View [/XYZ null null 0]
  /OUT pdfmark
[ /Title (Document information)
  /Page 2
  /View [/XYZ null null 0]
  /OUT pdfmark
[ /Title (Blah)
  /Page 3
  /View [/XYZ null null 0]
  /OUT pdfmark
[ /Title (Appendix)
  /Page 16
  /Count 4
  /View [/XYZ null null 0]
  /OUT pdfmark
    [ /Title (Sub1)
  /Page 17
  /Count 4
  /OUT pdfmark
    [ /Title (Subsub1)
  /Page 17
  /OUT pdfmark
    [ /Title (Subsub2)
  /Page 18
  /OUT pdfmark
    [ /Title (Subsub3)
  /Page 29
  /OUT pdfmark
    [ /Title (Subsub4)
  /Page 37
  /OUT pdfmark
    [ /Title (Sub2)
  /Page 40
  /OUT pdfmark
    [ /Title (Sub3)
  /Page 49
  /OUT pdfmark
    [ /Title (Sub4)
  /Page 56
  /OUT pdfmark
"""    
print('\r\n** Table of contents\r\n')
pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'
indent = 0
start = True
for title, page, count in re.findall(pattern, toc, re.DOTALL):
    title = (indent * ' ') + title
    count = int(count or 0)
    print(title.ljust(79, ".") + page.zfill(2))
    if count:
    count -= 1
    start = True
    if count and start:
    indent += 2
    start = False
    if not count and not start:
    indent -= 2
    start = True

This generates the following TOC, with subsub2 to subsub4 dedented one level 
too much:


** Table of contents

Title 
page.01
Document 
information...02
Blah...03
Appendix...16
  
Sub1.17
    
Subsub117
  
Subsub2..18
  
Subsub3..29
  
Subsub4..37
  
Sub2.40
  
Sub3.49
  
Sub4.56

What is the best approach to do this?

Thanks in advance!

Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Building a Package

[Albert-Jan Roskam]

On 21 Aug 2018 15:10, Glenn Schultz via Tutor  wrote:

All,
I have a project pthon(3.7) I have followed the python setup instructions. I 
get a dist folder with the correct files.  Now, I would like to install locally 
from my local machine (note I am not using virtual enviroment as our firewall 
does not allow installation of packages when using VE).  In my site packages 
directory I get the foo.dist-info but not foo folder.

Using setuptools and following the packaging python project I had to # out over 
70% of the example setup as they were all throwing warnings etc. and 
prohibiting the final build.

Is there an alternative source to guide one through project packaging? Its now 
been two-days of struggle with no success.

I would like to migrate some of my projects from R to Python.  But, honestly, 
packaging a python project seems far more difficult than building an R package. 
 I'm not trolling Python vs. R but my God, the level of frustration with python 
packaging is about to make me walk away.

Either I am a complete moron or packaging python is a poorly documented 
nightmare.  If anyone can point me to source for pyhton packaging that is 
better than the Packaging Python Projects website I would greatly appreciate 
the direction.

>> check out https://pypi.org//PyScaffold/ or 
https://github.com/audreyr/cookiecutter

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Questions about the formatting of docstrings

[Albert-Jan Roskam]

On 27 Jul 2018 06:34, boB Stepp  wrote:

I am near the end of reading "Documenting Python Code:  A Complete
Guide" by James Mertz, found at
https://realpython.com/documenting-python-code/  This has led me to a
few questions:

(1) The author claims that reStructuredText is the official Python
documentation standard.  Is this true?  If yes, is this something I
should be doing for my own projects?

(2) How would type hints work with this reStructuredText formatting?
In part of the author's reStructuredText example he has:

[...]
:param file_loc:  The file location of the spreadsheet
:type file_loc:  str
[...]

Hi Bob,

Have a look at numpydoc, which works with Sphinx. Numpydoc makes it much easier 
to read and write docstrings, while they can still be converted into nice 
looking docs, e.g html. See: 
https://codeandchaos.wordpress.com/2012/08/09/sphinx-and-numpydoc/

Albert-Jan

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iteration issues

[Albert-Jan Roskam]

Op 11 mei 2018 21:43 schreef Neil Cerutti <ne...@norwich.edu>:

On 2018-05-10, Albert-Jan Roskam <sjeik_ap...@hotmail.com> wrote:
> If a punctuation symbol is in your string:  Replace that symbol
> with an empty string.
>
>=>>> maybe something like
>
> import re
> no_interpunction = re.sub("[%s]" % re.escape(string.punctuation), '', 
> sentence)

str.translate can be used instead of re.

# Compute only once somewhere
punctuation_removal_table = str.maketrans({c: None for c in string.punctuation})


no_interpunction = sentence.translate(punctuation_removal_table)

Unfortunately you'll remove all the apostrophes and dashes, both
of which could be considered parts of words.

Nice, I did not think of str.translate. It appears to be the fastest (and most 
readable) method, though a compiled regex is also pretty fast: 
http://bbs.bugcode.cn/t/3320#IDb95ffe2791c1a59f0ac8175905705f34
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iteration issues

[Albert-Jan Roskam]

Verzonden vanaf mijn Samsung Galaxy-smartphone.

 Oorspronkelijk bericht 
Van: boB Stepp 
Datum: 10-05-18 21:53 (GMT+08:00)
Aan: tutor 
Onderwerp: Re: [Tutor] Iteration issues

On Wed, May 9, 2018 at 6:27 PM, Roger Lea Scherer  wrote:
> Hello, again.
>
> I want to count words in a text file. If a word repeats I want to increase
> the count by 1; if the word is new to the dictionary, I want to add the
> word to the dictionary. Everything works like I would like and expect,
> except for it only adds the last word of each line to the dictionary. What
> am I missing?
>
> import string
>
> file_name = 'oxford.txt'
> wordset = {}
> with open(file_name, 'r') as f:
> for line in f:
> sentence = line.strip()
> sentence = sentence.strip(string.punctuation)
> print(sentence)
> sentence = sentence.lower()
> word_list = sentence.strip()
> word_list = word_list.split(' ')
>
> for i in range(len(word_list)):
> word_list[i] = word_list[i].strip(string.punctuation)

I was wondering if you might want to write a small function so that
you can remove all punctuation symbols from each line in one fell
swoop?  Something like (in pseudocode):

Iterate over string.punctuation.
If a punctuation symbol is in your string:  Replace that symbol with
an empty string.



=>>> maybe something like

import re
no_interpunction = re.sub("[%s]" % re.escape(string.punctuation), '', sentence)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pandas read sql query advice

[Albert-Jan Roskam]

On Apr 28, 2018 06:54, Glenn Schultz  wrote:
>
> All,
>
> I have the following set-up (below) which will be used to call data from 
> multiple sectors.  There is a sql query (transact sql) and connection.  This 
> works fine.  However, I would like to parametrize the query so I can enter 
> different sectors.  I have checked through pyodbc and several SO posts.  
> However, I am stuck and cannot get the parametrized version to work.  Most 
> examples I have found are either sqlite or postgres.
>
> sql ="""select foo from mutable where model sector = 'mysector'"""
> conn = myconn
>
> def modeldata(modelquery, conn)
> modeldata = pandas.read_sql_query(sql, conn)
> return modeldata
>
> Here is what I have tried (various combinations) - what am I doing wrong?
>
> sql ="""select foo from mutable where model sector = ?""", [params]
> conn = myconn
>
> def modeldata(modelquery, conn, params)
> modeldata = pandas.read_sql_query(sql, conn, params)
> return modeldata

pandas.read_sql_query(sql, conn, params=params)

The third positional arg is actually index_col, so you need to use a keyword 
arg. 
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_query.html#pandas.read_sql_query

BTW, I think you can use a context manager to automatically close the 
connection:
with pyodbc.connect(your_conn_string) as conn:
df = pd.read_sql_query(sql, conn, params=params)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] XML Programs

[Albert-Jan Roskam]

On Apr 21, 2018 09:06, Peter Otten <__pete...@web.de> wrote:
>
> Glen wrote:
>
> > Thank you for your comprehensive reply. It was very helpful!
> > Just to clarify one point, is it not possible to search for a node
> > directly in the element / elementTree or do you first need to pull the
> > data into a class/dict?
>
> You certainly have other options
>
> >>> catalog.xpath("//catalog/book[contains(title,'Guide')]/title/text()")
> ["XML Developer's Guide", 'MSXML3: A Comprehensive Guide', 'Visual Studio 7:
> A Comprehensive Guide']
> >>> catalog.xpath("//catalog/book[starts-
> with(author,'Corets')]/title/text()")
> ['Maeve Ascendant', "Oberon's Legacy", 'The Sundered Grail']
>
> but then the question is whether you want to learn Python or XPath.

Neat! Can you recommend a good resource (book, webpage) for learning Xpath?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pythonic

[Albert-Jan Roskam]

Op 2 apr. 2018 15:31 schreef Steven D'Aprano <st...@pearwood.info>:
>
> On Mon, Apr 02, 2018 at 06:49:52AM -0600, Mats Wichmann wrote:
> > On 04/02/2018 02:56 AM, Alan Gauld via Tutor wrote:
> > > On 02/04/18 04:19, Steven D'Aprano wrote:
> > >> On Sun, Apr 01, 2018 at 10:58:51PM +0100, Alan Gauld via Tutor wrote:
> > >>> On01/04/18 20:20, Albert-Jan Roskam wrote:
> > >>>> fmt="%Y-%m-%d %H:%M\n"
> > >>>> f.write(now.strftime(fmt))
> > >>>> Lately I've been using format(), which uses __format__, because I find 
> > >>>> it slightly more readable:
> > >>>> format(datetime.now(), "%Y-%m-%d %H:%M")
> > >>> Interesting,
> > >>> I didn't know that format() recognised the datetime format codes.
> > >> It doesn't. It is the datetime object that recognises them. format()
> > >> merely passes the format string to the datetime.__format__ method, which
> > >> is what recognises the codes. It doesn't care what it is.
> > > Aha! That makes sense. I've never really used format() so have never
> > > bothered to find out how it works. To the point that until this thread I
> > > hadn't realized we even had a __format__() operator.
> > >
> > > As I said, I need to do some reading. Obviously a gap in my python
> > > education.
> > >
> >
> > so since we're all learning things here, how would this play out with
> > the new f-strings?
>
> I don't think f-strings are even a bit Pythonic.

"There should be one-- and preferably only one --obvious way to do it.":
1-str.format
2-% interpolation
3-string.Template
4-f strings
-...

I think that at least a few of these methods should become deprecated. Maybe 2 
and 3? (though I use 2 all the time!). Not sure about 4. A proper templating 
language like Jinja might be more desirable to have in the standard library, 
addition to a simple string substitution mechanism
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pythonic

[Albert-Jan Roskam]

On Mar 30, 2018 10:39, Alan Gauld via Tutor  wrote:
>
> On 30/03/18 03:48, Pat Martin wrote:
>
> > the "right" way to do it in python?
>
> More or less, a couple of comments below...
>
> > def Main():
>
> Python function names begin with a lowercase letter by convention.
>
> > """Run if run as a program."""
> > parser = argparse.ArgumentParser()
> ...
>
> >
> > now = datetime.datetime.now()
> > slug = args.title.replace(" ", "-").lower()
> >
> > with open("{}.md".format(slug), 'w') as f:
> > f.write("Title: {}\n".format(args.title))
> > f.write("Date: {}-{}-{} {}:{}\n".format(now.year,
> > now.month,
> > now.day,
> > now.hour,
> > now.minute))
>
> Formatting of dates and times is usually done using
> the time.strftime function which is specifically
> designed for that. It might be worth taking a peek
> at the docs on that one. You can call it directly
> on a datetime object - 'now' in your case:
>
> fmt="%Y-%m-%d %H:%M\n"
> f.write(now.strftime(fmt))

Lately I've been using format(), which uses __format__, because I find it 
slightly more readable:
format(datetime.now(), "%Y-%m-%d %H:%M")
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem with testing - container

[Albert-Jan Roskam]

Op 29 mrt. 2018 21:06 schreef "Shall, Sydney" :
>
> I have a problem with a specific test, I think.
>
> I use a Mac with OS X 10.13.3
> I use Anaconda with Python 3.5
>
> I have been writing tests for a Class that I have written. I am not
> finished yet, but there are already nearly 400 tests. The tests run and
> return OK. [After cleaning up many silly mistakes that I made.]
>
> However, now, when I run the tests (Unittest) I get the following warning:
> 
> lib/python3.5/unittest/case.py:1092: FutureWarning: elementwise
> comparison failed; returning scalar instead, but in the future will
> perform elementwise comparison
>if member in container:
> ...
>
> No other information is given.
>
> My problem is, how do I find out where exactly the problem originates.
> --

Try running Python with the option '-W error', or set PYTHONWARNINGS to 
'error'. https://docs.python.org/2/using/cmdline.html#envvar-PYTHONWARNINGS
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regex not working as desired

[Albert-Jan Roskam]

On Feb 27, 2018 09:50, Alan Gauld via Tutor  wrote:
>
> On 27/02/18 05:13, Cameron Simpson wrote:
>
> > hard to debug when you do. That's not to say you shouldn't use them, but 
> > many
> > people use them for far too much.
>
>
> > Finally, you could also consider not using a regexp for this particular 
> > task.
> > Python's "int" class can be called with a string, and will raise an 
> > exception
>
> And, as another alternative, you can use all() with a
> generator expression:
>
> >>> all(c.isdigit() for c in '1234')
> True
> >>> all(c.isdigit() for c in '12c4')
> False

I never understood why this is syntactically correct. It's like two parentheses 
are missing.

This I understand:
all((c.isdigit() for c in '12c4'))
Or this:
all([c.isdigit() for c in '12c4'])
Or this:
all((True, False))

But the way you wrote it, the generator expression just "floats" in between the 
parentheses that are part of the all() function. Is this something special 
about all() and any()?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about metaclasses

[Albert-Jan Roskam]

On Jan 10, 2018 19:32, Peter Otten <__pete...@web.de> wrote:
>
> Albert-Jan Roskam wrote:
>
> > Why does following the line (in #3)
>
> > # 3-
> > class Meta(type):
> > def __new__(cls, name, bases, attrs):
> > for attr, obj in attrs.items():
> > if attr.startswith('_'):
> > continue
> > elif not isinstance(obj, property):
> > import pdb;pdb.set_trace()
> > #setattr(cls, attr, property(lambda self: obj))  #
> > #incorrect!
> > raise ValueError("Only properties allowed")
> > return super().__new__(cls, name, bases, attrs)
> >
> > class MyReadOnlyConst(metaclass=Meta):
> > __metaclass__ = Meta
> > YES = property(lambda self: 1)
> > NO = property(lambda self: 0)
> > DUNNO = property(lambda self: 42)
> > THROWS_ERROR = 666
> >
> >
> > c2 = MyReadOnlyConst()
> > print(c2.THROWS_ERROR)
> > #c2.THROWS_ERROR = 777
> > #print(c2.THROWS_ERROR)
>
> > not convert the normal attribute int > a property?
> >
> > setattr(cls, attr, property(lambda self: obj))  # incorrect!
>
> cls is Meta itself, not MyReadOnlyConst (which is an instance of Meta).
> When the code in Meta.__new__() executes MyReadOnlyConst does not yet exist,
> but future attributes are already there, in the form of the attrs dict.
> Thus to convert the integer value into a read-only property you can
> manipulate that dict (or the return value of super().__new__()):
>
> class Meta(type):
> def __new__(cls, name, bases, attrs):
> for attr, obj in attrs.items():
> if attr.startswith('_'):
> continue
> elif not isinstance(obj, property):
> attrs[attr] = property(lambda self, obj=obj: obj)
>
> return super().__new__(cls, name, bases, attrs)
>
> class MyReadOnlyConst(metaclass=Meta):
> YES = property(lambda self: 1)
> NO = property(lambda self: 0)
> DUNNO = property(lambda self: 42)
> THROWS_ERROR = 666
>
> c = MyReadOnlyConst()
> try:
> c.THROWS_ERROR = 42
> except AttributeError:
> pass
> else:
> assert False
> assert c.THROWS_ERROR == 666

Thanks all for your replies!

Awesome, this is exactly what I want. I think I'll also override __setattr__  
so that each newly added attribute is automatically converted into a property
Is a metaclass the best/preferred/only way of doing this? Or is a class 
decorator an alternative route?

Is the following analogy for doing stuff when a class is created ('born') 
correct?
Metaclass --> prenatal surgery
__new__ --> perinatal surgery
Class decorator --> postnatal surgery

> PS: If you don't remember why the obj=obj is necessary:
> Python uses late binding; without that trick all lambda functions would
> return the value bound to the obj name when the for loop has completed.
> A simplified example:
>
> >>> fs = [lambda: x for x in "abc"]
> >>> fs[0](), fs[1](), fs[2]()
> ('c', 'c', 'c')
> >>> fs = [lambda x=x: x for x in "abc"]
> >>> fs[0](), fs[1](), fs[2]()
> ('a', 'b', 'c')
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about metaclasses

[Albert-Jan Roskam]

On Jan 10, 2018 18:57, Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Wed, Jan 10, 2018 at 04:08:04PM +0000, Albert-Jan Roskam wrote:
>
> > In another thread on this list I was reminded of
> > types.SimpleNamespace. This is nice, but I wanted to create a bag
> > class with constants that are read-only.
>
> If you expect to specify the names of the constants ahead of time, the
> best solution is (I think) a namedtuple.

Aaah *slaps forehead*, for some reason I didn't think about this, though I use 
namedtuples quite often. Using a metaclass for the very first time was great 
fun though :-)

> from collections import namedtuple
> Bag = namedtuple('Bag', 'yes no dunno')
> a = Bag(yes=1, no=0, dunno=42)
> b = Bag(yes='okay', no='no way', dunno='not a clue')
>
> ought to do what you want.
>
> Don't make the mistake of doing this:
>
> from collections import namedtuple
> a = namedtuple('Bag', 'yes no dunno')(yes=1, no=0, dunno=42)
> b = namedtuple('Bag', 'yes no dunno')(yes='okay', no='no way', dunno='not a 
> clue')

But if I do:
Bag = namedtuple('Bag', 'yes no dunno')
... and then I create hundreds of Bag instances, this doesn't have a large 
memory footprint, right? (Because of __slots__) Or is a regular tuple still 
(much) less wasteful?

> because that's quite wasteful of memory: each of a and b belong to a
> separate hidden class, and classes are rather largish objects.
>
>
> If you expect to be able to add new items on the fly, but have them
> read-only once set, that's a different story.
>
>
> --
> Steve
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When is and isn't "__file__" set?

[Albert-Jan Roskam]

On Jan 11, 2018 03:47, Steven D'Aprano  wrote:
>
> On Wed, Jan 10, 2018 at 08:02:24PM -0600, boB Stepp wrote:
>
> > I am still puzzling over things from the thread, "Why does
> > os.path.realpath('test_main.py') give different results for unittest
> > than for testing statement in interpreter?"  The basic question I am
> > trying to answer is how to determine the path to a particular module
> > that is being run.  For the experiments I have run thus far, the
> > module attribute, "__file__", has so far reliably given me the
> > absolute path to the module being run.  But the documentation suggests
> > that this attribute is optional.  So what can I rely on here with
> > "__file__"?  The first sentence of the cited quote is not illuminating
> > this sufficiently for me.
>
> Modules which are loaded from a .py or .pyc file on disk should always
> have __file__ set. If they don't, that's a bug in the interpreter.
>
> Modules which are loaded from a .dll or .so binary file also should have
> __file__ set.

And .pyd? I would hope so

> Modules that you create on the fly like this:
>
> py> from types import ModuleType
> py> module = ModuleType('module')
> py> module.__file__
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: module 'module' has no attribute '__file__'

Fascinating. Will this allow you to write a BLOB of the marshalled module 
object to a database? Do you happen to know a use case of this?

> will not have __file__ set unless you manually set it yourself. Such
> hand-made modules can be stored in databases and retrieved later, in
> which case they still won't have a __file__ attribute.
>
> Module objects which are built into the interpreter itself, like sys,
> also won't have a __file__ attribute:
>
> py> sys.__file__
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: module 'sys' has no attribute '__file__'
>
>
> One tricky, but unusual case, is that Python supports importing
> and running modules loaded from zip files. Very few people know this
> feature even exists -- it is one of Python's best kept secrets -- and
> even fewer know how it works. I'm not sure what happens when you load a
> module from a zip file, whether it will have a __file__ or not.

On this page https://pymotw.com/3/sys/imports.html, I see these lines:
"# Set a few properties required by PEP 302
mod.__file__ = fullname
"

Which suggests to me that it depends on the discipline of the custom importer's 
author. Since zipimport is part of the standard library, I would *guess* that 
they follow pep 302. Interesting to test it.

> Basically, if you still to reading module.__file__ for modules which
> come from a .py file, you should be absolutely fine. But to practice
> defensive programming, something like:
>
> try:
> path = module.__file__
> except AttributeError:
> print('handle the case where the module doesn't exist on disk')
> else:
> print('handle the case where the module does exist on disk')
>
>
> might be appropriate.
>
>
>
> --
> Steve
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] question about metaclasses

[Albert-Jan Roskam]

Hi,


In another thread on this list I was reminded of types.SimpleNamespace. This is 
nice, but I wanted to create a bag class with constants that are read-only. My 
main question is about example #3 below (example #2 just illustrates my thought 
process). Is this a use case to a metaclass? Or can I do it some other way 
(maybe a class decorator?). I would like to create a metaclass that converts 
any non-special attributes (=not starting with '_') into properties, if needed. 
That way I can specify my bag class in a very clean way: I only specify the 
metaclass, and I list the attributes as normal attrbutes, because the metaclass 
will convert them into properties.


Why does following the line (in #3) not convert the normal attribute into a 
property?

setattr(cls, attr, property(lambda self: obj))  # incorrect!



# 1-
# nice, but I want the constants to be read-only
from types import SimpleNamespace
const = SimpleNamespace(YES=1, NO=0, DUNNO=9)
const.YES = 0
print(const)


# 2-
# works, but I wonder if there's a builtin way
class Const(object):
"""Adding attributes is ok, modifying them is not"""
YES = property(lambda self: 1)
NO = property(lambda self: 0)
DUNNO = property(lambda self: 42)
#THROWS_ERROR = 666

def __new__(cls):
for attr in dir(cls):
if attr.startswith('_'):
continue
elif not isinstance(getattr(cls, attr), property):
raise ValueError("Only properties allowed")
return super().__new__(cls)
def __repr__(self):
kv = ["%s=%s" % (attr, getattr(self, attr)) for \
  attr in sorted(dir(Const)) if not attr.startswith('_')]
return "ReadOnlyNamespace(" + ", ".join(kv) + ")"

c = Const()
print(repr(c))
#c.YES = 42  # raises AttributeError (desired behavior)

print(c.YES)


# 3-
class Meta(type):
def __new__(cls, name, bases, attrs):
for attr, obj in attrs.items():
if attr.startswith('_'):
continue
elif not isinstance(obj, property):
import pdb;pdb.set_trace()
#setattr(cls, attr, property(lambda self: obj))  # incorrect!
raise ValueError("Only properties allowed")
return super().__new__(cls, name, bases, attrs)

class MyReadOnlyConst(metaclass=Meta):
__metaclass__ = Meta
YES = property(lambda self: 1)
NO = property(lambda self: 0)
DUNNO = property(lambda self: 42)
THROWS_ERROR = 666


c2 = MyReadOnlyConst()
print(c2.THROWS_ERROR)
#c2.THROWS_ERROR = 777
#print(c2.THROWS_ERROR)


Thank you in advance and sorry about the large amount of code!


Albert-Jan

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

From: eryk sun <eryk...@gmail.com>
Sent: Wednesday, January 10, 2018 3:56 AM
To: tutor@python.org
Cc: Albert-Jan Roskam
Subject: Re: [Tutor] Why does os.path.realpath('test_main.py') give different 
results for unittest than for testing statement in interpreter?
  

On Tue, Jan 9, 2018 at 2:48 PM, Albert-Jan Roskam
<sjeik_ap...@hotmail.com> wrote:
>
>> I think that it would be a great enhancement if os.realpath would return the 
>> UNC path if
>> given a mapped drive in Windows, if needed as extended path (prefixed with 
>> "\\?\UNC\").
>> That's something I use all the time, unlike symlinks, in Windows.
>
>pathlib can do this for you, or call os.path._getfinalpathname. 

I tried: 
>>> from os.path import _getfullpathname
>>> _getfullpathname(r"H:")
'h:\\path\\to\\folder'
>>> import os
>>> os.getcwd()
'h:\\path\\to\\folder'

I expected h:\ to be \\server\share\foo. The fact that the current working 
directory was returned was even more unexpected.

>>I recently helped someone that wanted the reverse, to map the resolved
>>UNC path back to a logical drive:

Oh dear. Why would anybody *want* the drive letters? They are only useful 
because (a) they save on keystrokes (b) they bypass the annoying limitation of 
the cd command on windows, ie. it does not work with UNC paths. 
Driveletter-->UNC conversion is useful when e.g. logging file paths. I do 
wonder whether the method used to assign the drive letter matters with the . I 
know net use, pushd, subst. I use 'net use' for more or less permanent drives 
and pushd/popd to get a temporary drive, available letter (cd nuisance).

>We can't assume in general that a user's special folders (e.g.
>Desktop, Documents) are in the default location relative to the
>profile directory. Almost all of them are relocatable. There are shell
>APIs to look up the current locations, such as SHGetKnownFolderPath.
>This can be called with ctypes [1]. For users other than the current
>user, it requires logging on and impersonating the user, which
>requires administrator access.
>
>[1]: https://stackoverflow.com/a/33181421/205580

Interesting code! I have used the following, which uses SHGetFolderPath, ie. 
without 'Known'.
from win32com.shell import shell, shellcon
desktop = shell.SHGetFolderPath(0, shellcon.CSIDL_DESKTOP, 0, 0)

Working with ctypes.wintypes is quite complex!


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

From: Tutor  on behalf of 
Steven D'Aprano 
Sent: Tuesday, January 9, 2018 8:47 AM
To: tutor@python.org
Subject: Re: [Tutor] Why does os.path.realpath('test_main.py') give different 
results for unittest than for testing statement in interpreter?

  
> The Python 3.5 source code for os.path.realpath under Windows looks like 
> this:
>
># realpath is a no-op on systems without islink support
> realpath = abspath

Déjà-vu [1], but I think that it would be a great enhancement if os.realpath 
would return the UNC path if given a mapped drive in Windows, if needed as 
extended path (prefixed with "\\?\UNC\"). That's something I use all the time, 
unlike symlinks, in Windows. 

And I would also welcome a convenience function in the os.path module that does 
expanduser, expandvars, normpath, realpath, and maybe more.

I haven't used pathlib yet. Will os.path development some day be frozen so 
pathlib becomes the standard? It seems to be nicer.

[1] http://python.6.x6.nabble.com/Tutor-getUncPath-mappedDrive-td4652304.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

On Jan 7, 2018 09:08, Steven D'Aprano  wrote:
>
> On Sun, Jan 07, 2018 at 12:49:59AM -0600, boB Stepp wrote:
>
> > Win7, Python 3.6.2
> >
> > If I run a unit test with the following embedded:
> >
> > print('realpath =', os.path.realpath('test_main.py'))
> >
> > I get the following in my test output (Only relevant line is shown):
> >
> > Ensure expected list of string integers is returned. ...
> > realpath = c:\Projects\solitaire_scorekeeper\test_main.py
>
>
> realpath() returns the canonical path of the given filename. It doesn't
> try to locate some actual existing file.

I always thought that os.path.realpath is the Python equivalent of Linux 
realpath/readlink (http://man7.org/linux/man-pages/man3/realpath.3.html). And 
that, thus, there's only a difference between input and output when the input 
is a symlink (and maybe also when it's a hard link - in this case the function 
actually also does something in Windows). But then, I don't really know the 
meaning of the word "canonical", to tell you the truth (maybe #4 in 
http://www.dictionary.com/browse/canonical)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Installing python and numpy on the Mac (OSX)

[Albert-Jan Roskam]

Op 23 dec. 2017 09:47 schreef Peter Hodges :
>
> Hi. I downloaded Python 3.6 from the python site, then followed online 
> directions for pip to install numpy (in users? —user was in the example).
> When I start IDLE in the Python 3.6 in Applications and then type import 
> numpy as np I get the following:
> import numpy as np
> Traceback (most recent call last):
>   File "", line 1, in 
> import numpy as np
> ModuleNotFoundError: No module named ‘numpy'
>
> Does this mean I need to set the $PATH with some new pathname?
> Or move the numpy directory into the python 3.6 directory?
> Or?

Or perhaps even easier: https://conda.io/docs/user-guide/install/download.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem in python online class

[Albert-Jan Roskam]

On Dec 21, 2017 09:58, Tim Cordsen via Tutor  wrote:
>
> Hello everybody,
> I am doing a python online class and I am lost. The "teacher" is a little
> chaotic and doesn't provide his code, so everyone must type on their own.
>
> Now the class reached a point where I am lost. It is about doing a simple
> web frontend with a form and saving the data of the form to mongodb.
>
> I have the form, but can't see the result in the console. Also there is
> nothing in mongodb after submitting.
>
> Is anybody willing to check my code and tell me where is my mistake?
>
> I am quite new to python, but not new to programming. I am
> mainframe-Cobol-programmer, so the basics of programming are known by me.

What framework are you using? Can you post the code of your web template, your 
view function, your form and your model? Have you tried using the pdb debugger 
(import pdb;pdb.set_trace())?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] restructuredtext's documentation

[Albert-Jan Roskam]

Op 12 dec. 2017 10:52 schreef Steven D'Aprano :
>
> On Mon, Dec 11, 2017 at 02:04:59PM +, adil gourinda wrote:
> > Hi
> > Please I have two questions:
> >
> > 1) Why restructuredtext's documentation is not included as a part of
> > the official Python's documentation?
>
> Because RestructuredText (ReST) is not part of Python.
>
> It is a project which uses Python.
>
>
> > 2) Can you generate a restructuredtext's PDF documentaion from
> > Docutils website(docutils.sourceforge.net/rst.html)?
>
> Sure. In your browser, go to the page you want, then choose Print from
> the browser File menu. When the printer dialog comes up, choose "Print
> To File", and set the options to PDF.

You could also try using a tool called pandoc, though you will also need 
LaTex/texlive for it (quite big). Alternatively: 
https://pypi.python.org/pypi/rst2pdf
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to test for the existence of a table in a sqlite3 db?

[Albert-Jan Roskam]

On Oct 16, 2017 15:12, Neil Cerutti  wrote:
>
> On 2017-10-15, boB Stepp  wrote:
> > Some things I am still pondering:
> >
> > 1)  If I adopt the incremental approach to creating and
> > initializing the working db, then it seems that the list,
> > "sql_scripts", should not be hard-coded into the program.  It
> > seems to me it should be off somewhere by itself with perhaps
> > other things that might evolve/change over time in its own file
> > where it (and its brethren) are easy to locate and update.
>
> An incremental approach is not recommended if you are using the
> sqlite3 module. In sqlite modifying table definitions is limited
> to the simple addition of a new row of data. Any other change
> requires you to create a new table, copy the old data into it,
> and then drop the old table.

Really? How so? I used ALTER TABLE ADD COLUMN the other day, which is part of 
this SQL dialect: https://sqlite.org/lang_altertable.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ctypes wintypes

[Albert-Jan Roskam]

(sorry for top-posting)

Perhaps this? https://pythonhosted.org/psutil/

From: Tutor  on behalf of 
Michael C 
Sent: Tuesday, October 3, 2017 9:30:43 PM
To: python tutor
Subject: [Tutor] ctypes wintypes

Hi all:

I am trying to create SYSTEM_INFO structure  and MEMORY_BASIC_INFORMATION
structure

I think there are modules for this purpose? Is it the ctypes.wintypes?

if so, please point me to a documentation for it.

Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] OT: how to best follow the posting style on this list

[Albert-Jan Roskam]

Hi,

I often read messages to this list from my Android from the mobile Hotmail 
page. On other occasions I would read from my browser, from a Windows or a 
Linux desktop. Howver, Hotmail pretty much sucks when it comes to obeying the 
posting style [1]. It's even worse than Yahoo mail, which I've also tried 
(until they got hacked). Is there a recommended way to do this?  Maybe gmail? I 
prefer to use more or less the same interface across platforms and devices, but 
this is not crucial. I've also been looking for pan-like apps (not necessarily 
free) for my Phone, but I couldn't find one that worked. Perhaps because I used 
a free news server (my ISP doesn't have one).
 
Thanks in advance!

Best wishes,
Albert-Jan

[1] https://en.wikipedia.org/wiki/Posting_style




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] logging to cmd.exe

[Albert-Jan Roskam]

Dear Mats, Peter and Eryk,

THANK YOU for your replies. What a wealth of information!

Have a great weekend!

Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] logging to cmd.exe

[Albert-Jan Roskam]

 From: Tutor <tutor-bounces+sjeik_appie=hotmail@python.org> on behalf of 
Peter Otten <__pete...@web.de>
 Sent: Monday, September 25, 2017 2:59 PM
 To: tutor@python.org
 Subject: Re: [Tutor] logging to cmd.exe
     
 Albert-Jan Roskam wrote:

 > Hi,
 > 
 > 
 > With Python 3.5 under Windows I am using the logging module to log
 > messages to stdout (and to a file), but this occasionally causes logging
 > errors because some characters cannot be represented in the codepage used
 > by cmd.exe (cp850, aka OEM codepage, I think). What is the best way to
 > prevent this from happening? The program runs fine, but the error is
 > distracting. I know I can use s.encode(sys.stdout.encoding, 'replace') and
 > log that, but this is ugly and tedious to do when there are many log
 > messages. I also don't understand why %r (instead of %s) still causes an
 > error. I thought that the character representation uses only ascii
 > characters?!

 Not in Python 3. You can enforce ascii with "%a":

 >>> euro = '\u20ac'
 >>> print("%r" % euro)
 '€'
 >>> print("%a" % euro)
 '\u20ac'



 > aaahh, I did not know about %a. Thank you! 


 Or you can set an error handler with PYTHONIOENCODING (I have to use 
 something that is not utf-8-encodable for the demo):

 $ python3 -c 'print("\udc85")'
 Traceback (most recent call last):
   File "", line 1, in 
 UnicodeEncodeError: 'utf-8' codec can't encode character '\udc85' in 
 position 0: surrogates not allowed

 $ PYTHONIOENCODING=:backslashreplace python3 -c 'print("\udc85")'
 \udc85


 > Nice to know about this variable, though I prefer not to change the 
environment because other will need to do the same. 
 For others who would like to read more: 
https://docs.python.org/3/using/cmdline.html


 Or you follow the convention and log to stderr:

 $ python3 -c 'import sys; print("\udc85", file=sys.stderr)'
 \udc85
 $ $ python3 -c 'import logging; logging.basicConfig(); 
 logging.getLogger().warn("\udc85")' > to_prove_it_s_not_stdout
 WARNING:root:\udc85


 > That's perhaps the best choice. But will messages with logging level 
warning and lower also be logged to stderr?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] logging to cmd.exe

[Albert-Jan Roskam]

From: Tutor <tutor-bounces+sjeik_appie=hotmail@python.org> on behalf of 
Mark Lawrence via Tutor <tutor@python.org>
Sent: Monday, September 25, 2017 4:19 PM
To: tutor@python.org
Subject: Re: [Tutor] logging to cmd.exe
    
On 25/09/2017 14:20, Albert-Jan Roskam wrote:
> Hi,
> 
> 
> With Python 3.5 under Windows I am using the logging module to log messages 
> to stdout (and to a file), but this occasionally causes logging errors 
> because some characters cannot be represented in the codepage used by cmd.exe 
> (cp850, aka OEM codepage, I think).  What is the best way to prevent this 
> from happening? The program runs fine, but the error is distracting. I know I 
> can use s.encode(sys.stdout.encoding, 'replace') and log that, but this is 
> ugly and tedious to do when there are many log messages. I also don't  
> understand why %r (instead of %s) still causes an error. I thought that the 
> character representation uses only ascii characters?!
> 
> 
> import logging
> import sys
> 
> assert sys.version_info.major > 2
> logging.basicConfig(filename="d:/log.txt", 
> level=logging.DEBUG,format='%(asctime)s %(message)s')
> handler = logging.StreamHandler(stream=sys.stdout)
> logger = logging.getLogger(__name__)
> logger.addHandler(handler)
> 
> s = '\u20ac'
> logger.info("euro sign: %r", s)
> 
> 
> 
> --- Logging error ---
> Traceback (most recent call last):
>    File "c:\python3.5\lib\logging\__init__.py", line 982, in emit
>  stream.write(msg)
>    File "c:\python3.5\lib\encodings\cp850.py", line 19, in encode
>  return codecs.charmap_encode(input,self.errors,encoding_map)[0]
> UnicodeEncodeError: 'charmap' codec can't encode character '\u20ac' in 
> position 12: character maps to 
> Call stack:
>    File "q:\temp\logcheck.py", line 10, in 
>  logger.info("euro sign: %r", s)
> Message: 'euro sign: %r'
> Arguments: ('\u20ac',)
> 
> 
> Thanks in advance for your replies!
> 
> 
> Albert-Jan
> 

Rather than change your code can you change the codepage with the chcp 
command?


> Good to keep in mind, but my objection would be similar to that with 
specifying the PYTHONIOENCODING variable: one needs to change the environment 
first before the script runs without errors.


C:\Users\Mark\Documents\MyPython>chcp
Active code page: 65001


> Wow! cmd.exe can use cp65001 aka utf8??? I always thought 65001 was 
not a 'real' codepage, even though some locales (e.g. Georgia) use it [1]. Why 
does cmd.exe still use cp850?

[1] https://docs.moodle.org/dev/Table_of_locales

PS: sorry about the missing quote (>>) markers. Hotmail can't do this. Is Gmail 
better?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] logging to cmd.exe

[Albert-Jan Roskam]

Hi,


With Python 3.5 under Windows I am using the logging module to log messages to 
stdout (and to a file), but this occasionally causes logging errors because 
some characters cannot be represented in the codepage used by cmd.exe (cp850, 
aka OEM codepage, I think). What is the best way to prevent this from 
happening? The program runs fine, but the error is distracting. I know I can 
use s.encode(sys.stdout.encoding, 'replace') and log that, but this is ugly and 
tedious to do when there are many log messages. I also don't understand why %r 
(instead of %s) still causes an error. I thought that the character 
representation uses only ascii characters?!


import logging
import sys

assert sys.version_info.major > 2
logging.basicConfig(filename="d:/log.txt", 
level=logging.DEBUG,format='%(asctime)s %(message)s')
handler = logging.StreamHandler(stream=sys.stdout)
logger = logging.getLogger(__name__)
logger.addHandler(handler)

s = '\u20ac'
logger.info("euro sign: %r", s)



--- Logging error ---
Traceback (most recent call last):
  File "c:\python3.5\lib\logging\__init__.py", line 982, in emit
stream.write(msg)
  File "c:\python3.5\lib\encodings\cp850.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u20ac' in position 
12: character maps to 
Call stack:
  File "q:\temp\logcheck.py", line 10, in 
logger.info("euro sign: %r", s)
Message: 'euro sign: %r'
Arguments: ('\u20ac',)


Thanks in advance for your replies!


Albert-Jan

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to write database-agnostic python code? (Is this even possible?)

[Albert-Jan Roskam]

(sorry for top posting)

There are various SQL abstraction layers for this. I have only used Sqlalchemy. 
This package has two APIs: query and ORM. You might want to look at the query 
API. This is closer to SQL than ORM. You can use straight(obj) to inspect the 
SQL that's emitted

From: Tutor  on behalf of boB 
Stepp 
Sent: Tuesday, September 12, 2017 7:05:39 PM
To: tutor
Subject: [Tutor] How to write database-agnostic python code? (Is this even 
possible?)

As I continue to read about SQL, one thing jumps out:  There are many
differences between how SQL statements are implemented among the
different database products.  Even for relatively simple,
straightforward things like field concatenation.  One DB might use
"||" as the operator.  Another uses "+".  Yet another only uses a
"CONCAT" function.  This is crazy!

It almost seems like I would need to write a DB-translator class that
takes a SQL statement (In some DB's version) and the target DB as
inputs, and then translates the statement into that DB's particular
usage of SQL.  So how does one write one's python program to be
DB-agnostic?  And if this is impossible, then what is the best way to
structure the overall program to isolate this SQL-specific stuff by
itself, insofar as possible, so any migrations to a new DB type is as
easy as possible?

--
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to create an object in database only if the object is not already there?

[Albert-Jan Roskam]

From: Tutor  on behalf of GMX 

Sent: Monday, September 11, 2017 9:38 AM
To: tutor@python.org
Subject: [Tutor] How to create an object in database only if the object is not 
already there?



Now when I create a Course object like this:

    >>> Course(title=‘A new course’)

An object is create in the database. I don’t want to have this behaviour, but 
what I want to be doing is create a Course object
and then only commit in the database if the course is not already available in 
the database. 


===> I have no experience with Pony, but with Sqlalchemy one could do something 
like:

if not session.query(Course.title).filter_by(title=‘A new course’).first():
course = Course(title=‘A new course’)

Or perhaps you could set a primary key constraint on course title.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [Python2.7] Convert (2,188,1) into (188,1)

[Albert-Jan Roskam]

From: Tutor [tutor-bounces+sjeik_appie=hotmail@python.org] on behalf of 
Alan Gauld via Tutor [tutor@python.org]
Sent: Tuesday, July 4, 2017 8:37 AM
To: tutor@python.org
Subject: Re: [Tutor] [Python2.7] Convert (2,188,1) into (188,1)

On 28/06/17 16:48, Allan Tanaka via Tutor wrote:
> Hi. I have array shape like: (2,188,1).

I assume this is a numpy array rather than the standard
library array? If so consider asking on the scipy/numpy
support forum for more specialised help.

> I want to make it like this: (188,1). I try that
> using .reshape(188,1)

Assuming this is a numpy array: If you want a 1D array you can use ravel 
(faster) or flatten (better name IMHO): 
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ravel.html
arr.reshape(2*188, 1) may also work.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Matplotlib in Tkinter

[Albert-Jan Roskam]

Hi,

(CCing Python Tutor again). I meant something like below. I am sending this 
from my phone, which is cumbersome to use an understatement, so the indentation 
is lost. But I hope you get the idea. You need to inherit from Frame.

Btw, this is useful if you would like to create an interactive graph: 
http://stackoverflow.com/questions/22052532/matplotlib-python-clickable-points

from Tkinter import *

import matplotlib
matplotlib.use("TkAgg")
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, 
NavigationToolbar2TkAgg
from matplotlib.figure import Figure

class PlotGraph(Frame):. # inherits from Frame
def __init__(self, root, xaxis, yaxis):
Frame.__init__(self, root, *args, **kwargs)  # execute __init__ from super class
self.root = root
root.title("Plotting a graph")
self.xaxis = xaxis
self.yaxis = yaxis
self.f = Figure(figsize = (5,5), dpi = 90)
self.a = self.f.add_subplot(111)
self.a.plot(self.xaxis,self.yaxis)

self.canvas = FigureCanvasTkAgg(self.f,self.root)
self.canvas.show()
self.canvas.get_tk_widget().pack(side = TOP, fill = BOTH, expand = True)

# Navigation Toolbar:
self.toolbar = NavigationToolbar2TkAgg(self.canvas,self.root)
self.toolbar.update()
self.canvas._tkcanvas.pack()

if __name__ == '__main__':
root = Tk()
root.title( "Creating a plot")
xaxis = [1,2,3,4,5,6]
yaxis = [7,8,9,10,11,12]
my_plot = PlotGraph(root, xaxis, yaxis, relief=SUNKEN) # you can add any of the 
arguments for the Frame initializer
my_plot.grid(row=0, column=0) # use grid here
button = Button (root, text="quit", fg='red', command=root.quit)
button.grid(row=1, column=0)
root.mainloop()

From: Pooja Bhalode <poojabhalod...@gmail.com>
Sent: Sunday, February 26, 2017 3:47:37 PM
To: Albert-Jan Roskam
Subject: Re: [Tutor] Matplotlib in Tkinter

Hi Albert,

Thank you so much for getting back to me. I tried your approach but it still 
fails for me. I created a class with pack(). But when I try using grid for the 
Label highlighted below, the gui hangs up and does not show anything.
Can you please shed more light on how to use my_plot as one unit and use grid 
packing for its instances?
Thank you so much.

Also, does using class with a pack layout help with screen blackening or two 
plots getting created?

Please let me know.
Yours truly,
Pooja

Code:

import matplotlib
matplotlib.use("TkAgg")
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, 
NavigationToolbar2TkAgg
from matplotlib.figure import Figure

from Tkinter import *
root = Tk()

Label(root, text = "Creating a plot").pack()

class PlotGraph:
def __init__(self, root, xaxis, yaxis):
self.root = root
root.title("Plotting a graph")
self.xaxis = xaxis
self.yaxis = yaxis
self.f = Figure(figsize = (5,5), dpi = 90)
self.a = self.f.add_subplot(111)
self.a.plot(self.xaxis,self.yaxis)


self.canvas = FigureCanvasTkAgg(self.f,self.root)
self.canvas.show()
self.canvas.get_tk_widget().pack(side = TOP, fill = BOTH, expand = True)

# Navigation Toolbar:
self.toolbar = NavigationToolbar2TkAgg(self.canvas,self.root)
self.toolbar.update()
self.canvas._tkcanvas.pack()

self.CloseButton = Button(root, text = "Close", command = root.quit)
self.CloseButton.pack()

xaxis = [1,2,3,4,5,6]
yaxis = [7,8,9,10,11,12]
my_plot = PlotGraph(root, xaxis, yaxis)
root.mainloop()

On Sun, Feb 26, 2017 at 1:22 AM, Albert-Jan Roskam 
<sjeik_ap...@hotmail.com<mailto:sjeik_ap...@hotmail.com>> wrote:
(Sorry for top-posting)

Hi, I recently ran into the same problem when I wanted to embed an interactive 
Matplotlib choropleth in my Tkinter app. I solved it by creating a separate 
class MyPlot for plot + toolbar (it inherited from Tkinter.Frame). MyPlot 
internally uses the pack geometry manager as given in the example from the mpl 
website. Then, you use MyPlot as 'one unit', and you use the grid method on its 
instances.

Best wishes,
Albert-Jan

From: Tutor 
<tutor-bounces+sjeik_appie=hotmail@python.org<mailto:hotmail@python.org>>
 on behalf of Pooja Bhalode 
<poojabhalod...@gmail.com<mailto:poojabhalod...@gmail.com>>
Sent: Sunday, February 19, 2017 7:36:32 PM
To: tutor@python.org<mailto:tutor@python.org>
Subject: [Tutor] Matplotlib in Tkinter

Hi,

I am trying to create a graph in Tkinter window. And following is a snipet
of the code.

Code:
  simroot = Toplevel(root)
simroot.title("Simulation of experiments")
Label(simroot, text = "Displaying concentration profiles with respect to
time:").pack(side = TOP)

Label(simroot, text = "Select the experiments to be displayed: ").pack(side
= TOP)
optionexp = ['Experiment 1', 'Experiment 2', 'Experiment 3', 'Experiment
4', 'Experiment 5']
expsim = StringVar()
dropexp = OptionMenu(simroot, expsim, *optionexp)
dropexp.pack(side = TOP)
dropexp.config(width = 15)

# Row 1 would print the name of the experiment se

Re: [Tutor] Matplotlib in Tkinter

[Albert-Jan Roskam]

(Sorry for top-posting)

Hi, I recently ran into the same problem when I wanted to embed an interactive 
Matplotlib choropleth in my Tkinter app. I solved it by creating a separate 
class MyPlot for plot + toolbar (it inherited from Tkinter.Frame). MyPlot 
internally uses the pack geometry manager as given in the example from the mpl 
website. Then, you use MyPlot as 'one unit', and you use the grid method on its 
instances.

Best wishes,
Albert-Jan

From: Tutor  on behalf of 
Pooja Bhalode 
Sent: Sunday, February 19, 2017 7:36:32 PM
To: tutor@python.org
Subject: [Tutor] Matplotlib in Tkinter

Hi,

I am trying to create a graph in Tkinter window. And following is a snipet
of the code.

Code:
  simroot = Toplevel(root)
simroot.title("Simulation of experiments")
Label(simroot, text = "Displaying concentration profiles with respect to
time:").pack(side = TOP)

Label(simroot, text = "Select the experiments to be displayed: ").pack(side
= TOP)
optionexp = ['Experiment 1', 'Experiment 2', 'Experiment 3', 'Experiment
4', 'Experiment 5']
expsim = StringVar()
dropexp = OptionMenu(simroot, expsim, *optionexp)
dropexp.pack(side = TOP)
dropexp.config(width = 15)

# Row 1 would print the name of the experiment selected and the species
displayed would be in row3.
Label(simroot, text = "Select concentration species:").pack(side = TOP)
concsim = StringVar()
optionlistsim = ['A', 'B', 'C', 'D', 'E']
dropsim = OptionMenu(simroot, concsim, *optionlistsim)
dropsim.pack(side = TOP)
dropsim.config(width = 8)
Label(simroot, text = "").pack(side = LEFT)

def Submitplot():
print "Create plot"

f = Figure(figsize = (5,5), dpi = 80)
a = f.add_subplot(111)
a.plot([1,2,3,4,5],[10,11,12,14,15])

canvas = FigureCanvasTkAgg(f, simroot)
canvas.show()
canvas.get_tk_widget().pack(side = BOTTOM)

toolbar = NavigationToolbar2TkAgg(canvas, simroot)
toolbar.update()
canvas._tkcanvas.pack(side = BOTTOM)
Button(simroot, text = "Submit and Plot", command = Submitplot).pack(side =
BOTTOM)

Here, the output comes as:
[image: Inline image 1]
The problem is I am not able to use grid layout instead of pack (used
here). I want to place the text to the left, top side in the window.

Also, when I hover the mouse over the figure and the x and y values are
shown, it flickers really fast and the white space in the Tkinter window
turns black when it flickers. Can someone please tell me what the issue
could be? Also, the toolbar should have been at the bottom but it shows up
at the top.

I am using pack in this Toplevel of the main root whereas in the rest of
the code, I am using grid layout. I switched to pack for this
snipet because I could not figure out how grid would work for this part.

Can someone please let me know how to correct these issues?
Thankyou so much.

Pooja
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] ReadableInt

[Albert-Jan Roskam]

Hi,


Why does the call to str() below return '1' and not 'one'? Should I implement 
__new__ because int is immutable? I have a datafile that contains values and an 
associated metadata file with the associated labels. I need to recode the 
values. Because just using the values is a bit error prone I would like to see 
the associated labels while debugging. How stupid is the code below on a scale 
from 1-10? Remember I only intend to use it while debugging the given script.


Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (In
tel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> class ReadableInt(int):
... def __str__(self):
... return {1: "one", 2: "two"}.get(self, self)
...
>>> builtins.int = ReadableInt
>>> str(1)  # I expected 'one'
'1'

>>> builtins.int.__str__




Btw, initially I hoped to be able to do this with the (newish) pandas dtype 
'category', but unless

I am overlooking something, but this does not seem possible:


>>> import pandas as pd
>>> pd.Categorical.from_codes([1, 2], ["one", "two"], ordered=True)
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Users\Albert-Jan\AppData\Local\Programs\Python\Python35-32\lib\site-p
ackages\pandas\core\categorical.py", line 471, in from_codes
raise ValueError("codes need to be between -1 and "
ValueError: codes need to be between -1 and len(categories)-1
>>> pd.Categorical.from_codes([0, 1], ["one", "two"], ordered=True)
[one, two]
Categories (2, object): [one < two]


Happy coding in 2017!


Best wishes,

Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Array of Tkinter objects?

[Albert-Jan Roskam]

Hi,

I have created a grid of Tkinter Entry object that looks a bit like a sheet in 
MS Excel. The data come from a pandas.DataFrame. I use a nested loop, one for 
rows and one for cols. Inside the loop, I create the Entry instances which I 
also grid() right away.

This works nicely, though a little slow. What data structure would be suitable 
so I can store the instances in it? I want to be able to easily call grid() on 
all of them. Also, I want to easily call .configure() on subsets, for instance 
to change the background color of a row when a certain value in that row is too 
high/low.

A numpy array of np.objects might be okay, but it is called NUMpy for a reason, 
no? Still, the way of indexing seems very useful/intuitive/neat here. Likewise 
for pd.DataFrame. Other options might be a list of lists, or a dictionary of 
the form {(row, col): Entry instance}.

Thanks in advance for your replies!

ALBERT-JAN
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __init__

[Albert-Jan Roskam]

From: Tutor <tutor-bounces+sjeik_appie=hotmail@python.org> on behalf of 
Steven D'Aprano <st...@pearwood.info>
Sent: Thursday, September 1, 2016 1:24 AM
To: tutor@python.org
Subject: Re: [Tutor] __init__
    
On Wed, Aug 31, 2016 at 06:35:44PM +0000, Albert-Jan Roskam wrote:
> 
> >In Python 3, old-style classes are gone. Even in Python 2, they're 
> >discouraged.
> 
> This suggests that old-style classes are still used, even in Python 3, 
> doesn't it?
> 
> albertjan@debian:~$ python3.5
> Python 3.5.0 (default, Apr 13 2016, 20:39:27) 
> [GCC 4.9.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import re, inspect
> >>> re.findall("class.+:", inspect.getsource(tkinter))
> ['class Event:', 'class Variable:', 'class StringVar(Variable):', 'class 
> IntVar(Variable):', 'class DoubleVar(Variable):', 'class 
> BooleanVar(Variable):', 'class Misc:', 'className):', 'class(self):', 
> 'class(self, className, sequence=None, func=None, add=None):',  'class(self, 
> className, sequence):', 'class CallWrapper:', 'class XView:', 'class YView:', 
> 'class Wm:', 'class Tk(Misc, Wm):', 'className):', 'class_tcl):', 
> 'class_py):', "className='Tk', useTk=0):", 'class Pack:', 'class Place:', 
> 'class Grid:', 'class BaseWidget(Misc):',  'classes:', 'classes:', 'class 
> Widget(BaseWidget, Pack, Place, Grid):', 'class Toplevel(BaseWidget, Wm):', 
> 'class Button(Widget):', 'class Canvas(Widget, XView, YView):', 'class 
> Checkbutton(Widget):', 'class Entry(Widget, XView):', 'class Frame(Widget):', 
>  "class_' in cnf:", "class' in cnf:", 'class Label(Widget):', 'class 
> Listbox(Widget, XView, YView):', 'class Menu(Widget):', 'class 
> Menubutton(Widget):', 'class Message(Widget):', 'class Radiobutton(Widget
 ):', 'class Scale(Widget):', 'class Scrollbar(Widget):', 'class Text(Widget, 
XView, YView):', 'class _setit:', 'class OptionMenu(Menubutton):', 'class 
Image:', 'class PhotoImage(Image):', 'class BitmapImage(Image):', 'class 
Spinbox(Widget, XView):', 'class  LabelFrame(Widget):', 'class 
PanedWindow(Widget):']


No. A bare class with no declared parent has object automatically added, 
in Python 3.

Compare the output of this:

class K:
    pass

print(K.__bases__)


in Python 2 and Python 3, and you will see () in Python 2 (an empty 
tuple, i.e. no bases classes) but (,) in Python 3.


=> Aha, thank you, I did not know that. So for Python-3-only code, the 
idiom is "class SomeClass:", and for Python2/3 code "class SomeClass(object)".
What are the most common code changes a developer needs to make when switching 
from old- to new-style classes?



> I sometimes want to use the @property decorator in classes that 
> inherit from e.g. tkinter.Frame. The getter then works, but the setter 
> fails without any sign. So I then also inherit from object, as in
> 
> class WickedFrame(tkinter.frame, object):
[...]
> I do this in Python 2.7. Is this the recommended approach? I also like the 
> fact that I can use super() that way.

I believe that is fine. 


===> thanks again :-)



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] capture output from a subprocess while it is being produced

[Albert-Jan Roskam]

Hi,

I have a Tkinter program where I can fire up a commandline program which could 
run for quite a while (let's say 15 minutes or more). While the program is 
running, I would like to show the output that I normally see in the terminal 
(actually, the target platform is windows but now I am on Linux) in a text 
entry widget. The main purpose of this is to show that "something is 
happening". 

Anyway, below I tried to make a simplified version, without Tkinter. I fire up 
the commandline process in a separate thread, and I try to append lines of the 
output to a deque. The 'ls' command returns 2964 lines on my system. So I 
expect the deque to contain less items, because every time the while loop in 
process() sleeps, it would miss a few items. But it doesn't! It's like the ls 
output is returned as one chunk.

What am I doing wrong?

Thanks!

Albert-Jan


from subprocess import Popen, PIPE
from threading import Thread
from collections import deque
import sys
import time
import os.path


def process(dq):
"""Run a commandline program, with lots of output"""
cwd = os.path.dirname(sys.executable)
proc = Popen("ls -lF", cwd=cwd, shell=True, stdout=PIPE, stderr=PIPE)  
#print len(proc.stdout.readlines())  # 2964
output = errors = None
while True:
try:
#output, errors = proc.communicate() # "Wait for process to 
terminate."  
output = proc.stdout.read()  # docs: "Use communicate() rather than 
.stdin.write, .stdout.read or .stderr.read"
errors = proc.stderr.read()
proc.stdout.flush()
except ValueError:
break
finally:
if errors:
raise RuntimeError(str(errors))

if output:
dq.append(output)
time.sleep(0.0001)
if not output:
break

def update(dq):
"""This is supposed to return a new, not necessarily contiguous, chunk of 
output"""
if dq:
result = dq.pop()
print time.asctime().center(79, "*")
print result
return result

if __name__ == "__main__": 

dq = deque()
output_thread = Thread(target=process, args=(dq,), name="output_thread")
#output_thread.setDaemon(True)
output_thread.start()


lines = []
while output_thread.is_alive():
update(dq)
time.sleep(0.1)
if not dq: 
break
for item in dq:
lines += item.splitlines()
output_thread.join()
print len(lines)
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Urgent: unicode problems writing CSV file

[Albert-Jan Roskam]

> Date: Thu, 9 Jun 2016 03:57:21 +1000
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] Urgent: unicode problems writing CSV file
> 
> On Wed, Jun 08, 2016 at 01:18:11PM -0400, Alex Hall wrote:



>> 
>> csvWriter.writerow([info.encode("utf8") if type(info)is unicode else info
>> for info in resultInfo])
> 
> Let's break it up into pieces. You build a list:
> 
> [blah blah blah for info in resultInfo]
> 
> then write it to the csv file with writerow. That is straightforward.
> 
> What's in the list?
> 
> info.encode("utf8") if type(info)is unicode else info
> 
> So Python looks at each item, `info`, decides if it is Unicode or not, 
> and if it is Unicode it converts it to a byte str using encode, 
> otherwise leaves it be.
> 
> If it makes you feel better, this is almost exactly the solution I would 
> have come up with in your shoes, except I'd probably write a helper 
> function to make it a bit less mysterious:
> 
> def to_str(obj):
> if isinstance(obj, unicode):
> return obj.encode('uft-8')
> elif isinstance(obj, str):
> return obj
> else:
> & Or maybe an error?
> return repr(obj)
> 
> csvWriter.writerow([to_string(info) for info in resultInfo])
>

The docs also offer some code for working with Unicode/CSV, see the bottom of 
this page: 
https://docs.python.org/2/library/csv.html


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Getting started in testing

[Albert-Jan Roskam]

(sorry for top-posting)

I liked 
https://www.packtpub.com/application-development/python-testing-beginners-guide 
though perhaps it was a bit too basic. It covers forest, unit test, nose.

> Date: Thu, 19 May 2016 12:34:27 -0700
> From: carr...@tjc.com
> To: tutor@python.org
> Subject: [Tutor] Getting started in testing
> 
> Is anyone aware of any good tutorials on testing one's Python code?
> 
> These days, I'm a hobby programmer, writing little things just for my own 
> use, and don't really sweat testing much. But I do have one niche 
> open-source project where I need to be a bit more regimented, and 
> specifically need to develop a set of tests to be passed before releasing.
> 
> Any resources would be helpful. I am aware of the docs on unittest, but 
> I'm looking for a more tutorial approach.
> 
> I use Python 2.7, and my project is a module with no GUI, if that matters. 
> Thanks.
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about __copy__ and __deepcopy__

[Albert-Jan Roskam]

> From: oscar.j.benja...@gmail.com
> Date: Thu, 14 Apr 2016 21:34:39 +0100
> Subject: Re: [Tutor] question about __copy__ and __deepcopy__
> To: sjeik_ap...@hotmail.com
> CC: tutor@python.org
> 
> On 14 April 2016 at 20:38, Albert-Jan Roskam <sjeik_ap...@hotmail.com> wrote:
> > Hi,
> >
> > Lately I have been using the "mutable namedtuple"  shown below a lot. I 
> > found it somewhere on StackOverflow or ActiveState or something.
> > In its original form, it only had an __init__ method.
> 
> I don't know about your copy/deepcopy stuff. It looked fine to me but
> I'm not very experienced in that area. I wonder what this mutable
> namedtuple is for though.
> 
> Looking at it:
> 
> > class Record(dict):
> >
> > def __init__(self, *args, **kwargs):
> > super(Record, self).__init__(*args, **kwargs)
> > self.__dict__ = self
> 
> It's not so much a mutable namedtuple as an "attribute dict". It's a
> dict whose keys can be accessed as attributes - because it is its own
> instance dict. Of course it also has dict attributes like pop, items
> etc. (see dir(dict) for a list). The reason that dicts use d[k] rather
> than d.k for accessing keys is to be able to store arbitrary keys
> without conflicting with the dict's methods which are attributes.
> 
> A namedtuple is something very different. It subclasses tuple and the
> whole idea is that an instance is lightweight (not having an attribute
> dict) and hashable and imutable etc. A mutable version of that might
> just look like:
> 
> class Point(object):
> # No instance dict:
>  __slots__ = ['x', 'y']
> 
> def __init__(self, x, y):
> self.x = x
> self.y = y
> 
> It depends what you really want it for though.

HI Oscar, 

Ok, I agree that "mutable namedtuple" is a misnomer. I started using a regular 
namedtuple, and then I wanted something with similar functionality, but with 
the possibility to update/mutate the fields. The AttrDict was the option I 
liked. The order of the fields is rarely important in my case. I can easily use 
the AttrDict with SqlAlchemy, too. What I like about both namedtuple and 
AttrDict is attribute lookup: that makes code so, so, s much easier to 
read. This seems to be a nice generalization of your code:

class Point(object):

def __init__(self, **kwargs):
Point.__slots__ = kwargs.keys()
for k, v in kwargs.items():
setattr(self, k, v)

point = Point(x=1, y=2, z=3)
print point.x
point.x = 42
print point.x


I have no experience with __slots__ (but I think I roughly know what it's for). 
I expected this to fail:

point.remark = "booh"

point
Out[36]: <__main__.Point at 0x7f282de9ccd0>

point.remark
Out[37]: 'booh'

How can it be that __slots__ may be extended once the Point class has been 
instantiated?

(sorry if this post is a bit long --this is just so much fun!) If I add a 
convenience getter/setter, the setter does indeed only seem to work for 
attributes that were in __slots__ already (here "z")

class Point(object):

def __init__(self, **kwargs):
Point.__slots__ = kwargs.keys()
for k, v in kwargs.items():
setattr(self, k, v)

@property
def values(self):
return {attr: getattr(self, attr) for attr in Point.__slots__}
@values.setter
def values(self, d):
for attr, value in d.items():
print "setting", attr, value
setattr(self, attr, value)


point = Point(x=1, y=2, z=3)
print point.x
point.x = 42
print point.x
print point.values
point.values = dict(a=1, b=2, c=3)
print point.values  ## no a, b, c here!
point.values = dict(a=1, b=2, z=444)
print point.values   # z is updated, though!


Thank you!

Albert-Jan

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about __copy__ and __deepcopy__

[Albert-Jan Roskam]

> Date: Fri, 15 Apr 2016 16:30:16 +1000
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] question about __copy__ and __deepcopy__
> 
> On Thu, Apr 14, 2016 at 07:38:31PM +, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > Lately I have been using the "mutable namedtuple"  shown below a lot. 
> > I found it somewhere on StackOverflow or ActiveState or something. In 
> > its original form, it only had an __init__ method. I noticed that 
> > copying Record objects sometimes failed.
> 
> Failed how?
> 
> Given how simple the class looks, I wonder whether that's a bug in copy. 
> Apart from a fancy __str__ method, there's practically nothing to it!
> 
> So I took your Record class, stripped out the __copy__ and __deepcopy__ 
> methods, created a slightly complex instance, and tried copying it:
> 
> d = Record(spam=23, ham=42, eggs=[], cheese={})
> d.cheese[1] = None
> d.cheese[2] = ['a', 'b', 'c']
> d.eggs.append(100)
> d.eggs.append(200)
> d.eggs.append(d)
> d.eggs.append(d.cheese)
> 
> from copy import copy, deepcopy
> 
> copy(d)
> deepcopy(d)
> 
> 
> and both appeat to work correctly. So perhaps you ought to look more 
> carefully at the copying failure?

Hi Steven,

Heh, it's my fancy __str__ method that confused me. This is what I get when I 
run my code without __copy__ and __deepcopy__
runfile('/home/albertjan/Downloads/attrdict_tutor.py', 
wdir='/home/albertjan/Downloads')
{'x': 1, 'y': 2, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} {'x': 1, 'y': 2, 'z': [0, 
1, 2, 3, 4, 5, 6, 7, 8, 9]}
{'x': 1, 'y': 2, 'z': [42, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} {}   # print 
statements --> call __str__

dc.__str__()
Out[19]: '{}'

dc.__repr__()
Out[20]: "{'y': 2, 'x': 1, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}"

Wicked. I do not understand the output of dc.__str__(). Somehow self.__dict__ 
is empty. If I store a deepcopy of the instance __dict__ in a class attribute 
"Record.dict_copy", __str__ does not return just the two curly braces. It's 
wasteful to create a copy, though.

from copy import copy, deepcopy

class Record(dict):

def __init__(self, *args, **kwargs):
super(Record, self).__init__(*args, **kwargs)
self.__dict__ = self

def __str__(self):
#items = ["%r: %r" % (k, v) for k, v in sorted(self.__dict__.items())]
Record.dict_copy = deepcopy(self)  # store it as a class attribute.
items = ["%r: %r" % (k, v) for k, v in sorted(Record.dict_copy.items())]
return "{" + ", ".join(items) + "}"

runfile('/home/albertjan/Downloads/attrdict_tutor.py', 
wdir='/home/albertjan/Downloads')
{'x': 1, 'y': 2, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} {'x': 1, 'y': 2, 'z': [0, 
1, 2, 3, 4, 5, 6, 7, 8, 9]}
{'x': 1, 'y': 2, 'z': [42, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} {'x': 1, 'y': 2, 'z': 
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}

dc.__str__()
Out[28]: "{'x': 1, 'y': 2, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}"

dc.__repr__()
Out[29]: "{'y': 2, 'x': 1, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}"



  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] question about __copy__ and __deepcopy__

[Albert-Jan Roskam]

Hi,

Lately I have been using the "mutable namedtuple"  shown below a lot. I found 
it somewhere on StackOverflow or ActiveState or something.
In its original form, it only had an __init__ method. I noticed that copying 
Record objects sometimes failed. So I implemented __copy__ and __deepcopy__,
Is this the correct way to do this? In my original use case, even shallow 
copies failed, but I can't reproduce that anymore (or maybe I am imagining 
things!). 
Is __copy__ really needed here? Does __deepcopy__ make any sense at all? My 
tests pass, but still I am not sure!

Thanks!

Albert-Jan

from copy import copy, deepcopy

class Record(dict):

    def __init__(self, *args, **kwargs):
    super(Record, self).__init__(*args, **kwargs)
    self.__dict__ = self

    def __str__(self):
    items = ["%r: %r" % (k, v) for k, v in sorted(self.__dict__.items())]
    return "{" + ", ".join(items) + "}"

    def __copy__(self):
    return Record(**self.__dict__.copy())

    def __deepcopy__(self, memo):
    address = id(self)
    try:
    return memo[address]
    except KeyError:
    memo[address] = {k: copy(v) for k, v in self.items()}
    return deepcopy(self, memo)

if __name__ == "__main__":
    
    # only shallow needed    
    record = Record(x=1, y=2, z=3)
    cp = copy(record)
    assert record == cp and not record is cp
    record = Record(x=1, y=2, z=3)
    dc = deepcopy(record)
    assert record == dc and not record is dc
    
    # mutable value: deepcopy needed
    L = range(10)
    record = Record(x=1, y=2, z=L)
    cp = copy(record)
    print record, cp
    assert record == cp and not record is cp
    dc = deepcopy(record)
    assert record == dc and not record is dc
    L.insert(0, 42)
    expect = {'y': 2, 'x': 1, 'z': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
    assert dc == expect and not record is dc
    print record, dc
    
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Alan G Week 10 warmup assignment help

[Albert-Jan Roskam]

> To: tutor@python.org
> From: alan.ga...@btinternet.com
> Date: Sat, 2 Apr 2016 09:08:04 +0100
> Subject: Re: [Tutor] Alan G Week 10 warmup assignment help
> 
> On 02/04/16 02:21, Daniella Sapozhnikova wrote:
> > I changed the file to this:
> 
> > DATA = {2: 7493945,
> > 76: 4654320,
> ...
> > 153: 12074784,
> > 8: 4337229}
> > 
> > def iter_dict_funky_sum(data=DATA):
> > funky = 0
> > for key, value in data.iteritems():
> > funky += value - key
> > return funky
> > 
> > 
> > however now pylint is returning:
> > task_07.py:36: [W0102(dangerous-default-value), iter_dict_funky_sum]
> > Dangerous default value DATA (__builtin__.dict) as argument
> > 
> > how can I pass the dictionary into the function, since pylint considers the
> > wway I'm doing it to be wrong?
> > 
> 
> I would drop the default value for the parameter.
> ( I suspect that Python doesn't like DATA as a default value
> because it means you cant use the function anywhere DATA is
> not defined. But I'm not sure about that)

I thought this warning was about the use of mutable  default arguments and its 
counterintuitive behavior: 
https://pythonconquerstheuniverse.wordpress.com/2012/02/15/mutable-default-arguments/
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __str__ vs. sys.displayhook

[Albert-Jan Roskam]

> Date: Mon, 21 Mar 2016 02:50:17 +1100
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] __str__ vs. sys.displayhook
>
> On Sun, Mar 20, 2016 at 01:53:04PM +, Albert-Jan Roskam wrote:
>> Hi,
>>
>> The other day I was playing with this to make a convenient string version of 
>> a named tuple:
>>
>>>>> from collections import namedtuple as nt
>>>>> Record = nt("Record", "x y z")
>>>>> Record.__str__ = lambda self: "| %s |" % " | ".join([item + ": " + 
>>>>> str(getattr(self, item)) for item in self._fields])
>>>>> str(Record(1, 2, 3))
>> '| x: 1 | y: 2 | z: 3 |'
>>
>> Then, after reading another discussion on the main Python list 
>> (https://www.mail-archive.com/python-list@python.org/msg409771.html) I 
>> realized I could do the same thing with a displayhook.
>> When would I (re)implement __str__, and when would I redefine 
>> sys.displayhook? Does the displayhook also affect the way things are logged 
>> (with the ' logging'  module)? It feels like sys.displayhook is more " heavy 
>> artillery",. but I might be wrong about that.
>
> As the documentation says, "sys.displayhook is called on the result of
> evaluating an expression entered in an interactive Python session." So
> the answer to your question is, you would redefine sys.displayhook when
> you want to control how objects are displayed in the interactive Python
> shell.
>
> https://docs.python.org/3/library/sys.html#sys.displayhook
>
> When you run a script, sys.displayhook is never called. It is only
> called when you are in interactive mode, typing at the Python prompt.


It seems that ipython uses a displayhook in conjunction with pprint.pprint. Or 
at least something to that effect. That is *very* convenient, esp. with 
unordered objects like dictionaries.


> When you assign a value to a variable:
>
> the_string = repr(some_expression)
>
> sys.displayhook is never called.
>
> sys.displayhook has nothing to do with the repr or str conversion,
> except in the sense that the display hook will probably *use* the
> value's repr or str. For instance, here's a rather silly display hook in
> action:

aha, so it's an implementation detail of the hook to often use str or repr. But 
not a requirement.


> py> def hook(value):
> ... if value is None: return
> ... sys.stdout.write("+++REDACTED+++\n")
> ...
> py> sys.displayhook = hook
> py> 23 + 1
> +++REDACTED+++
> py> 1000*2
> +++REDACTED+++
> py> "hello".upper()
> +++REDACTED+++
>
> But even with this display hook in place, the repr and str of values are
> unchanged. It's just tricky to see them unless you bypass the hook:
>
> py> print(23 + 1) # print directly to the screen without calling hook
> 24
>
> You will re-define __str__ when you want to control how objects are
> converted to strings using the str() function, and __repr__ for how
> objects are converted using the repr() function.
>
> The intention is that repr() returns the "string representation" of the
> object ("what does it look like?") while str() "converts this object to
> a string". They are usually the same, but not always:
>
> py> from fractions import Fraction
> py> x = Fraction(1, 2)
> py> print(str(x))
> 1/2
> py> print(repr(x))
> Fraction(1, 2)

That makes sense to me. I read somewhere [1, 2]) that __repr__, when 
implemented and when possible, should return an eval-able representation of the 
object (though this is not enforce/checked). Or as somebody put it: __str__ is 
about readability, and __repr__ is about eval-ability:
eval(repr(obj)) == obj, e.g.

>>> d = {1: 10}
>>> eval(repr(d)) == d
True

[1] https://docs.python.org/2/reference/datamodel.html#object.__repr__
[2] 
http://stackoverflow.com/questions/1436703/difference-between-str-and-repr-in-python
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] __str__ vs. sys.displayhook

[Albert-Jan Roskam]

Hi,

The other day I was playing with this to make a convenient string version of a 
named tuple:

>>> from collections import namedtuple as nt
>>> Record = nt("Record", "x y z")
>>> Record.__str__ = lambda self: "| %s |" % " | ".join([item + ": " + 
>>> str(getattr(self, item)) for item in self._fields])
>>> str(Record(1, 2, 3))
'| x: 1 | y: 2 | z: 3 |'

Then, after reading another discussion on the main Python list 
(https://www.mail-archive.com/python-list@python.org/msg409771.html) I realized 
I could do the same thing with a displayhook.
When would I (re)implement __str__, and when would I redefine sys.displayhook? 
Does the displayhook also affect the way things are logged (with the ' logging' 
 module)? It feels like sys.displayhook is more " heavy artillery",. but I 
might be wrong about that.

Thanks in advance for your replies!

Best wishes,
Albert-Jan



  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Changing the interpreter prompt symbol from ">>>" to ???

[Albert-Jan Roskam]

< snip lots of useful info >

Hi all, 

Thanks a lot for all your replies. I am whole lot wiser now :)

Best wishes,
Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Changing the interpreter prompt symbol from ">>>" to ???

[Albert-Jan Roskam]

> From: eryk...@gmail.com
> Date: Sun, 13 Mar 2016 13:58:46 -0500
> Subject: Re: [Tutor] Changing the interpreter prompt symbol from ">>>" to ???
> To: tutor@python.org
> CC: sjeik_ap...@hotmail.com
> 
> On Sun, Mar 13, 2016 at 3:14 AM, Albert-Jan Roskam
> <sjeik_ap...@hotmail.com> wrote:
> > I thought that utf-8 (cp65001) is by definition (or by design?) impossible
> > for console output in windows? Aren't there "w" (wide) versions of functions
> > that do accept utf-8?
> 
> The wide-character API works with the native Windows character
> encoding, UTF-16. Except the console is a bit 'special'. A surrogate
> pair (e.g. a non-BMP emoji) appears as 2 box characters, but you can
> copy it from the console to a rich text application, and it renders
> normally. 

That is very useful to know.

> The console also doesn't support variable-width fonts for
> mixing narrow and wide (East Asian) glyphs on the same screen. If that
> matters, there's a program called ConEmu that hides the console and
> proxies its screen and input buffers to drive an improved interface
> that has flexible font support, ANSI/VT100 terminal emulation, and
> tabs. If you pair that with win_unicode_console, it's almost as good
> as a Linux terminal, but the number of hoops you have to go through to
> make it all work is too complicated.


So windows uses the following (Western locales):
console: cp437 (OEM codepage)
"bytes": cp1252 (ANSI codepage)
unicode: utf-16-le (is 'mbcs' equivalent to utf-16-*?)


Sheesh, so much room for errors. Why not everything utf-8, like in linux? 
Is cmd.exe that impopular that Microsoft does not replace it with something 
better?
I understand that this silly OEM codepage is a historical anomaly, but am I 
correct
in saying that the use of codepages (with stupid differences such as latin-1 vs 
cp1252 as a bonus)
are designed to hamper cross-platform compatibility (and force people to stick 
with windows)?

 
> Some people try to use UTF-8 (codepage 65001) in the ANSI API --
> ReadConsoleA/ReadFile and WriteConsoleA/WriteFile. But the console's
> UTF-8 support is dysfunctional. It's not designed to handle it.
> 
> In Windows 7, WriteFile calls WriteConsoleA, which decodes the buffer
> to UTF-16 using the current codepage and returns the number of UTF-16
> 'characters' written instead of the number of bytes. This confuses
> buffered writers. Say it writes a 20-byte UTF-8 string with 2 bytes
> per character. WriteFile returns that it successfully wrote 10
> characters, so the buffered writer tries to write the last 10 bytes
> again. This leads to a trail of garbage text written after every
> write.


Strange. I would have thought it writes the first 10 bytes (5 characters) and 
that the remaining 10 bytes end up in oblivion.


> When a program reads from the console using ReadFile or ReadConsoleA,
> the console's input buffer has to be encoded to the target codepage.
> It assumes that an ANSI character is 1 byte, so if you try to read N
> bytes, it tries to encode N characters. This fails for non-ASCII
> UTF-8, which has 2 to 4 bytes per character. However, it won't
> decrease the number of characters to fit in the N byte buffer. In the
> API the argument is named "nNumberOfCharsToRead", and they're sticking
> to that literally. The result is that 0 bytes are read, which is
> interpreted as EOF. So the REPL will quit, and input() will raise
> EOFError.
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Pmw/Tkinter question

[Albert-Jan Roskam]

> Date: Sun, 13 Mar 2016 19:19:55 -0500
> From: robertvst...@gmail.com
> To: tutor@python.org
> Subject: Re: [Tutor] Pmw/Tkinter question
> 
> I just cannot leave this problem alone; however, I *do* believe I have
> found the correct way to do what Albert-Jan desires within the
> publicly accessible features of Pmw and Tkinter.  I looked at the Pmw
> documentation online at
> 
> http://pmw.sourceforge.net/doc/OptionMenu.html
> 
> and it said:
> 
> 
> menubutton
> 
> The menu button displaying the currently selected value. By default,
> this component is a Tkinter.Menubutton.
> 
> 
> So I looked up Tkinter.Menubutton next in one of my books and it has
> the desired anchor option.  So if we add
> 
>  menubutton_anchor = 'w'
> 
> in the appropriate place, then I get the desired alignment.  BTW, this
> fits in with Peter's suggestion.  It just needs to be using the normal
> Pmw/Tkinter machinery:

Thanks, you're right! After the line:
self.vege_menu.grid(row=1, column=0, sticky = 'w', padx = 10, pady = 10)

either one of the following lines does the trick:
self.vege_menu.configure(menubutton_anchor=Tkinter.W)
#self.vege_menu.configure(menubutton_anchor=' w')
#self.vege_menu._menubutton.config(anchor=Tkinter.W)

This does not work, however:
self.vege_menu.grid(row=1, column=0, sticky = 'w', padx = 10, pady = 
10, menubutton_anchor=Tkinter.W)

I tested this on debian linux jessie, with Python 2.7 (it turned out that I 
incorrectly compiled 3.5, no _tkinter)

Best wishes,
Albert-Jan




  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Pmw/Tkinter question

[Albert-Jan Roskam]

> Date: Sat, 12 Mar 2016 23:08:20 -0600
> Subject: Re: [Tutor] Pmw/Tkinter question
> From: robertvst...@gmail.com
> To: sjeik_ap...@hotmail.com
> CC: tutor@python.org
>
> On Sat, Mar 12, 2016 at 2:17 AM, Albert-Jan Roskam
> <sjeik_ap...@hotmail.com> wrote:
>> Hello,
>>
>> Below is a slightly modified example about Pmw/Tkinter from 
>> http://pmw.sourceforge.net/doc/howtouse.html.
>> I changed the geometry manager from 'pack' to 'grid', because I use that in 
>> all other parts of my program.
>> The problem is, 'broccoli' won't display nicely under vege_menu. I want it 
>> to left-align, not center (I don't case that it's truncated)...
>
> Not having run your code (I did not have Pmw installed.), I think I
> misunderstood your question(s). I went ahead and installed Pmw and
> ran your code. I don't have Py 2 installed, so I had to do the usual
> conversions to make it Py 3 compatible. Once I did that I got all of
> your labels showing up on the right side of the menu buttons. I
> presume you want them on the left side of each button? To do that you
> with the grid manager, you do not want the buttons to attach to the
> left side of the overall megawidget. Instead, allow the menu buttons
> to center in each row by omitting the sticky option from grid() and
> then your labels have room to attach on the left side. Assuming that
> I am now trying to solve the correct problem, my Py *3* code is:
>
> # -*- coding: utf-8 -*-
>
> import tkinter
> import Pmw
>
> class Demo:
> def __init__(self, parent):
> # Create and pack the OptionMenu megawidgets.
> # The first one has a textvariable.
> self.var = tkinter.StringVar()
> self.var.set('steamed')
> self.method_menu = Pmw.OptionMenu(parent,
> labelpos = 'w',
> label_text = 'Choose method:',
> menubutton_textvariable = self.var,
> items = ['baked', 'steamed', 'stir fried', 'boiled', 'raw'],
> menubutton_width = 10,
> )
> self.method_menu.grid(row=0, column=0, padx = 10, pady = 10)
>
> self.vege_menu = Pmw.OptionMenu (parent,
> labelpos = 'w',
> label_text = 'Choose vegetable:',
> items = ('broccoli', 'peas','carrots', 'pumpkin'),
> menubutton_width = 10,
> command = self._printOrder,
> )
> self.vege_menu.grid(row=1, column=0, padx = 10, pady = 10)
>
> self.direction_menu = Pmw.OptionMenu (parent,
> labelpos = 'w',
> label_text = 'Menu direction:',
> items = ('flush', 'above', 'below', 'left', 'right'),
> menubutton_width = 10,
> command = self._changeDirection,
> )
> self.direction_menu.grid(row=2, column=0, padx = 10, pady = 10)
>
> menus = (self.method_menu, self.vege_menu, self.direction_menu)
> Pmw.alignlabels(menus)
>
> def _printOrder(self, vege):
> # Can use 'self.var.get()' instead of 'getcurselection()'.
> print('You have chosen %s %s.' % \
> (self.method_menu.getcurselection(), vege))
>
> def _changeDirection(self, direction):
> for menu in (self.method_menu, self.vege_menu, self.direction_menu):
> menu.configure(menubutton_direction = direction)
>
> if __name__ == "__main__" :
> root = Pmw.initialise()
> root.title("Demo")
> widget = Demo(root)
> root.mainloop()
>
> I omitted the extra "broccoli" words, to clarify the alignment issues.
> I hope I am going in the direction you wanted! Nonetheless, having
> never used the Python megawidgets before, I found this educational.
> In passing, I was surprised that it does not matter about the case of
> the sticky positions. I know I have gotten errors at work by not
> having the proper case for these, but then, I wasn't doing classes.

Hi Bob,

Thanks! However, if you replace 'broccoli' with a much longer option, you will 
see what I meant with the alignment problem. One can no longer read the first 
letters of the option once it's selected.
That's kinda ugly IMHO. Peter's suggestion solves it (but it uses a 
private/undocumented method). 

Pmw seems to be quite handy, compared to Tkinter. Not sure how it compares to 
tix. I suppose both offer a lot of shortcuts for many common tasks. I've only 
wprked with it very briefly, so I havent discovered much of it. One thing liked 
is the ability to easily use validators in text entry widgets, with red colors 
for invalid values etc.

Still, tkinter and friends will never win a beauty contest. PyQt is much more 
visually attractive. I was surprised motivating this could be. Qt Designer 
makes the process easier, but tkinter is more straightforward.

Best wishes,
Albert-Jan

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Changing the interpreter prompt symbol from ">>>" to ???

[Albert-Jan Roskam]

> From: eryk...@gmail.com
> Date: Sun, 13 Mar 2016 01:39:11 -0600
> To: tutor@python.org
> Subject: Re: [Tutor] Changing the interpreter prompt symbol from ">>>" to ???
> 
> On Sat, Mar 12, 2016 at 12:46 AM, boB Stepp  wrote:
> > I did with the non-printing control character, but not with '\u25ba' !
> >  So I had to go through some contortions after some research to get my
> > Win7 cmd.exe and PowerShell to display the desired prompt using
> 
> The console is hosted by another process named conhost.exe. When
> Python is running in the foreground, the cmd shell is just waiting in
> the background until Python exits.
> 
> > '\u25ba' as the character with utf-8 encoding.  My new
> > pythonstartup.py file (Which PYTHONSTARTUP now points to) follows:
> >
> > #!/usr/bin/env python3
> >
> > import os
> > import sys
> >
> > os.system('chcp 65001')# cmd.exe and PowerShell require the code
> > page to be changed.
> > sys.ps1 = '\u25ba '  # I remembered to add the additional space.
> 
> chcp.com calls SetConsoleCP (to change the input codepage) and
> SetConsoleOutputCP. You can call these functions via ctypes if you
> need to separately modify the input or output codepages. For example:

I thought that utf-8 (cp65001) is by definition (or by design?) impossible for 
console output in windows? Aren't there "w" (wide) versions of functions that 
do accept utf-8? 

Best wishes,
Albert-Jan

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] OT: (Continuous) integration testing: Can a solo developer with very limited resources truly do this?

[Albert-Jan Roskam]

> From: d...@hashcollision.org
> Date: Sat, 12 Mar 2016 18:34:02 -0800
> To: robertvst...@gmail.com
> Subject: Re: [Tutor] OT: (Continuous) integration testing: Can a solo 
> developer with very limited resources truly do this?
> CC: tutor@python.org
> 
> On Sat, Mar 12, 2016 at 10:10 AM, boB Stepp  wrote:
> > From "Practices of an Agile Developer" by Venkat Subramaniam and Andy
> > Hunt, c. 2006, page 90:
> >
> > 
> > You're already writing unit tests to exercise your code.  Whenever you
> > modify or refactor your code, you exercise your test cases before you
> > check in the code.  All you have to do now is exercise your test cases
> > on each supported platform or environment.
> 
> 
> Hi Bob,
> 
> 
> There are some common infrastructure that folks can use.  For example,
> Travis CL is one such continuous integration system that's getting
> traction:
> 
>  https://travis-ci.org/
> 
> I think there's another CI system called Jenkins CI that's also used
> quite a bit.

I have used Jenkins CI. It was designed for/with Java, but works very well with 
Python. I used a multi-configuration project, where you run all unittests in a 
matrix of N python versions * M operating systems. There are many, many plugins 
for Jenkins CI and I believe I used the VirtualBox and the Git plugins. 
You can set things up in such a way that, with each commit, Jenkins tells 
Virtualbox to fire up the various OSes, then tell Git to make a shallow clone 
of your repo, then run the tests for each python version (I think this works 
with tox and virtualenv). It takes a while before everything is up and running, 
but you can start by setting up Jenkins so it just runs the tests on your host 
OS. Then you gradually add one OS after another. Hackintosh (unofficial MacOS) 
was the hardest to get working. Later it got annoying that all these VMs were 
fired up after each commit. So I manually fired them up whenever I believed it 
was a good moment. At one point, my tests failed after I updated Jenkins 
itself, which was very, very annoying. Jenkins, VirtualBox and Git are free 
software.

Appveyor is a similar service to Travis CI, but for windows. It is free for 
open source projects. You can set it up such that all unittests (and possibly 
build steps) are run after each "git push".

Docker is a way to efficiently run multiple Linux instances. I have no 
experience with this (because I don't want to limit things to Linux).

Vagrant is a disgustingly easy way to fire up VMs (http://www.vagrantbox.es/). 
You can create your own vagrant boxes with Veewee. I now see that there is a 
vagrant plugin for virtualbox 
(https://wiki.jenkins-ci.org/display/JENKINS/Vagrant-plugin)

Hercules is a program to emulate mainframes (http://www.hercules-390.org/). It 
felt to geeky to try and combine this with Jenkins :-).

One very different point: I once read somewhere that requirements.txt should 
"pin the versions": http://nvie.com/posts/pin-your-packages/.
That seems a very good idea to me. 


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Pmw/Tkinter question

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Sat, 12 Mar 2016 10:28:36 +0100
> Subject: Re: [Tutor] Pmw/Tkinter question
> 
> Albert-Jan Roskam wrote:
> 
> > Hello,
> > 
> > Below is a slightly modified example about Pmw/Tkinter from
> > http://pmw.sourceforge.net/doc/howtouse.html. I changed the geometry
> > manager from 'pack' to 'grid', because I use that in all other parts of my
> > program. The problem is, 'broccoli' won't display nicely under vege_menu.
> > I want it to left-align, not center (I don't case that it's truncated). In
> > plain Tkinter I could do "self.vege_menu.configure(anchor=Tkinter.W)" but
> > this raises a KeyError in Pmw.
> 
> I don't see an official way to to change the alignment, but looking into the 
> source there's
> 
> class OptionMenu(Pmw.MegaWidget):
> 
> def __init__(self, parent = None, **kw):
> 
> [...]
>   self._menubutton = self.createcomponent('menubutton',
>   (), None,
>   Tkinter.Menubutton, (interior,),
>   borderwidth = 2,
>   indicatoron = 1,
>   relief = 'raised',
>   anchor = 'c',
>   highlightthickness = 2,
>   direction = 'flush',
> takefocus = 1,
>   )
> 
> so if you don't mind touching _private attributes
> 
> > self.vege_menu = Pmw.OptionMenu (parent,
> > labelpos = 'w',
> > label_text = 'Choose vegetable:',
> > items = ('broccoli-broccoli-broccoli-broccoli', 'peas',
> > 'carrots', 'pumpkin'), menubutton_width = 10,
> > command = self._printOrder,
> > )
>   self.vege_menu._menubutton.config(anchor=Tkinter.W)
> 
> > #self.vege_menu.pack(anchor = 'w', padx = 10, pady = 10)
> > self.vege_menu.grid(row=1, column=0, sticky = 'w', padx = 10, pady
> > = 10)
> 
> should work.

Ahh, yes, that works! Thank you!! I should indeed have looked at the source 
myself.
The Tk website is often informative, but not everything is exposed by 
Tkinter/Pmw.
This is also the first time I tried Pmw. A lot less verbose than Tkinter, but 
it takes some time
to learn the names/meaning of various parameters. They differ from the 
equivalent params in 
Tkinter and are often more descriptive, but pre-existing knowledge of Tkinter 
creates some "noise".

Best wishes,
Albert-Jan


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Pmw/Tkinter question

[Albert-Jan Roskam]

Hello,

Below is a slightly modified example about Pmw/Tkinter from 
http://pmw.sourceforge.net/doc/howtouse.html.
I changed the geometry manager from 'pack' to 'grid', because I use that in all 
other parts of my program.
The problem is, 'broccoli' won't display nicely under vege_menu. I want it to 
left-align, not center (I don't case that it's truncated). In plain Tkinter I 
could do "self.vege_menu.configure(anchor=Tkinter.W)" but this raises a 
KeyError in Pmw.

The real reason for switching from Tkinter to Pmw is that Pmw is more compact, 
and it has an 'index' method that I could use to link two optionmenus.

Any ideas? Hope this is ok to ask as Pmw is a layer on top of Tkinter.

Thank you!
Albert-Jan


# -*- coding: utf-8 -*-

import Tkinter
import Pmw

class Demo:
def __init__(self, parent):
# Create and pack the OptionMenu megawidgets.
# The first one has a textvariable.
self.var = Tkinter.StringVar()
self.var.set('steamed')
self.method_menu = Pmw.OptionMenu(parent,
labelpos = 'w',
label_text = 'Choose method:',
menubutton_textvariable = self.var,
items = ['baked', 'steamed', 'stir fried', 'boiled', 'raw'],
menubutton_width = 10,
)
#self.method_menu.pack(anchor = 'w', padx = 10, pady = 10)
self.method_menu.grid(row=0, column=0, sticky = 'w', padx = 10, pady = 
10)

self.vege_menu = Pmw.OptionMenu (parent,
labelpos = 'w',
label_text = 'Choose vegetable:',
items = ('broccoli-broccoli-broccoli-broccoli', 'peas', 
'carrots', 'pumpkin'),
menubutton_width = 10,
command = self._printOrder,
)
#self.vege_menu.pack(anchor = 'w', padx = 10, pady = 10)
self.vege_menu.grid(row=1, column=0, sticky = 'w', padx = 10, pady = 10)

self.direction_menu = Pmw.OptionMenu (parent,
labelpos = 'w',
label_text = 'Menu direction:',
items = ('flush', 'above', 'below', 'left', 'right'),
menubutton_width = 10,
command = self._changeDirection,
)
#self.direction_menu.pack(anchor = 'w', padx = 10, pady = 10)
self.direction_menu.grid(row=2, column=0, sticky = 'w', padx = 10, pady 
= 10)

menus = (self.method_menu, self.vege_menu, self.direction_menu)
Pmw.alignlabels(menus)

def _printOrder(self, vege):
# Can use 'self.var.get()' instead of 'getcurselection()'.
print 'You have chosen %s %s.' % \
(self.method_menu.getcurselection(), vege)

def _changeDirection(self, direction):
for menu in (self.method_menu, self.vege_menu, self.direction_menu):
menu.configure(menubutton_direction = direction)

if __name__ == "__main__" :
root = Pmw.initialise()
root.title("Demo")
widget = Demo(root)
root.mainloop()   
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Recommendations for best tool to write/run Python :p:

[Albert-Jan Roskam]

> Date: Thu, 3 Mar 2016 15:54:56 -0600
> From: da...@graniteweb.com
> To: tutor@python.org
> Subject: Re: [Tutor] Recommendations for best tool to write/run Python :p:
> 
> * Alan Gauld  [2016-03-03 11:02]:
> > On 03/03/16 09:31, Thomas C. Hicks wrote:
> > > On 03/03/2016 02:26 AM, Lisa Hasler Waters wrote:
> > >> Could you please recommend the best Python tools for writing and running
> > >> our code for the long term? Also, we are hoping to find free tools!
> > >>
> > > Most people on this list are a lot smarter than me so there are probably 
> > > good reasons for it but I have used Ipython (now Jupyter) for teaching 
> > > my kids programming in middle and high school.
> > 
> > IPython is great as an interactive environment but the OP
> > specifically mentioned writing longer programs and editing
> > files which is not what IPython does best. I suspect that's
> > why it didn't get a mention earlier.
> 
> Very likely, but it's definitely worth mentioning as a runtime environment.
> It's a big step above the basic built-in CLI

Yes, IPython is extremely useful, even if only for testing code snippets. 
IPython Notebook is worth mentioning as well.
My favourite IDE currently is Spyder (free). PyScripter is nice (and free) too, 
but I don't like the way it behaves with pdb (with a little menu). Both come 
with Python(x, y). PyCharm is great too (maybe even nicer than Spyder), but 
it's only free for open source projects. It is Java-based, but only Sun Java 
and not Open JDK (it occasionally frooze with Open JDK so I stoped using it).

Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] lc_ctype and re.LOCALE

[Albert-Jan Roskam]

> From: eryk...@gmail.com
> Date: Fri, 29 Jan 2016 07:14:07 -0600
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> To: tutor@python.org
> CC: sjeik_ap...@hotmail.com
> 
> On Thu, Jan 28, 2016 at 2:23 PM, Albert-Jan Roskam
> <sjeik_ap...@hotmail.com> wrote:
> > Out of curiosity, I wrote the throw-away script below to find a character 
> > that is classified
> > (--> LC_CTYPE) as digit in one locale, but not in another.
> 
> The re module is the wrong tool for this. The re.LOCALE flag is only
> for byte strings, and in this case only ASCII 0-9 are matched as
> decimal digits. It doesn't call the isdigit() ctype function. Using
> Unicode with re.LOCALE is wrong. 


Ok, good to know. In my original Python-2 version of the script I did convert 
the ordinal to a byte string, but it was still a utf-8 byte string.



> The current locale doesn't affect the
> meaning of a Unicode character. Starting with 3.6 doing this will
> raise an exception.


I find it strange that specifying either re.LOCALE or re.UNICODE is still the 
"special case". IMHO it is a historical anomaly that ASCII is the "normal 
case". Matching accented characters should not require any special flags.


> The POSIX ctype functions such as isalnum and isdigit are limited to a
> single code in the range 0-255 and EOF (-1). For UTF-8, the ctype
> functions return 0 in the range 128-255 (i.e. lead bytes and trailing
> bytes aren't characters). Even if this range has valid characters in a
> given locale, it's meaningless to use a Unicode value from the Latin-1
> block, unless the locale uses Latin-1 as its codeset.
> 
> Python 2's str uses the locale-aware isdigit() function. However, all
> of the locales on my Linux system use UTF-8, so I have to switch to
> Windows to demonstrate two locales that differ with respect to
> isdigit(). 


In other words: the LC_CTYPE is only relevant with codepage encodings?


You could use PyWin32 or ctypes to iterate over all the
> locales known to Windows, if it mattered that much to you.
> 
> The English locale (codepage 1252) includes superscript digits 1, 2, and 3:
> 
> >>> locale.setlocale(locale.LC_CTYPE, 'English_United Kingdom')
> 'English_United Kingdom.1252'
> >>> [chr(x) for x in range(256) if chr(x).isdigit()]
> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3', '\xb9']
> >>> unicodedata.name('\xb9'.decode('1252'))
> 'SUPERSCRIPT ONE'
> >>> unicodedata.name('\xb2'.decode('1252'))
> 'SUPERSCRIPT TWO'
> >>> unicodedata.name('\xb3'.decode('1252'))
> 'SUPERSCRIPT THREE'


Is character classification also related to the compatibility form of unicode 
normalization?
>>> unicodedata.normalize("NFKD", u'\xb3')
u'3'

(see also http://unicode.org/reports/tr15/)


> Note that using the re.LOCALE flag doesn't match these superscript digits:
> 
> >>> re.findall(r'\d', '0123456789\xb2\xb3\xb9', re.LOCALE)
> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']



Ok, now I am upset. I did not expect this at all! I would have expected the re 
results to be in line with the str.isdigit results. If LC_CTYPE is not relevant 
isn't "re.DIACRITIC"  a better name for the re.LOCALE flag?


 
> The Windows Greek locale (codepage 1253) substitutes "Ή" for superscript 1:
> 
> >>> locale.setlocale(locale.LC_CTYPE, 'Greek_Greece')
> 'Greek_Greece.1253'
> >>> [chr(x) for x in range(256) if chr(x).isdigit()]
> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3']
> 
> >>> unicodedata.name('\xb9'.decode('1253'))
> 'GREEK CAPITAL LETTER ETA WITH TONOS'

Ok, I switched to Windows to see this with my own eyes. Checked the regex. 
Strange, but fun to know.

Thanks a lot for your thorough reply!
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html

[Albert-Jan Roskam]

> Date: Fri, 29 Jan 2016 14:50:57 +
> Subject: [Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html

sorry, I did not intend to send this to Python Tutor (the link is awesome, 
though)
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] lc_ctype and re.LOCALE

[Albert-Jan Roskam]

> From: oscar.j.benja...@gmail.com
> Date: Fri, 29 Jan 2016 16:32:57 +
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> To: sjeik_ap...@hotmail.com
> CC: tutor@python.org
> 
> On 28 January 2016 at 20:23, Albert-Jan Roskam <sjeik_ap...@hotmail.com> 
> wrote:
> >
> > Out of curiosity, I wrote the throw-away script below to find a character 
> > that is classified (--> LC_CTYPE) as digit in one locale, but not in 
> > another.
> > I ran it with 5000 locale combinations in Python 2 but did not find any 
> > (somebody shut down my computer!). I just modified the code so it also
> > runs in Python 3. Is this the correct way to find such locale-dependent 
> > regex matches?
> 
> Eryk already gave you a better explanation of the locale stuff than I
> could but I have a separate comment about the algorithmic performance
> of your code (since you mentioned that it took a long time).
> 
> You're looping over all pairs of locales:
> 
> ...
> > for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 2), >
> ...
> > for i in xrange(sys.maxunicode + 1):   # 1114111
> > s = unichr(i)  #.encode("utf8")
> > try:
> > locale.setlocale(locale.LC_CTYPE, locale1)
> > m1 = bool(regex.match(s))
> > locale.setlocale(locale.LC_CTYPE, locale2)
> > m2 = bool(regex.match(s))
> > if m1 ^ m2:  # m1 != m2
> 
> Suppose there are N locales and M is sys.maxunicode. The number of
> pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
> loop over M characters so the innermost loop body is repeated
> something like M*N**2 times.
> 
> Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
> your code above. We can swap the loops around so that the outer loop
> is over unicode characters. Then the inner loop can be over the
> locales but we only loop over all N locales once rather than over all
> N**2 pairs of locales. This looks like this:
> 
> for c in unicode_chacters:
> matched = f(locales[0], c) # Check the first locale
> for locale in locales:
> assert all(f(locale, c) == matched for locale in locales)
> 
> This way you call f(locale, c) M*N times which if N is not small
> should be a lot faster than M*N**2 times.

Hi Oscar,

Putting the Busiest Loop on the InsideWhen you have nested loops, think about 
which loop you want on the outside and which you want on the inside. Following 
is an example of a nested loop that can be improved:I blindly followed a code 
tuning tip I once read about in Code Complete (McConnel 2004; page 623 [1]):
" Putting the Busiest Loop on the Inside

When you have nested loops, think about which loop you want on the outside and
which you want on the inside. Following is an example of a nested loop that can 
be
improved:
...
The key to improving the loop is that the outer loop executes much more often 
than the
inner loop. Each time the loop executes, it has to initialize the loop index, 
increment it
on each pass through the loop, and check it after each pass" 

[1] 
https://khmerbamboo.files.wordpress.com/2014/09/code-complete-2nd-edition-v413hav.pdf

Your advice makes perfect sense, though. So McConnel's code tuning tip may be a 
rule-of-thumb, with exceptions, right?

Thanks!


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] lc_ctype and re.LOCALE

[Albert-Jan Roskam]

> From: sjeik_ap...@hotmail.com
> To: oscar.j.benja...@gmail.com
> Date: Sun, 31 Jan 2016 19:56:21 +
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> CC: tutor@python.org
> 
> 
> > From: oscar.j.benja...@gmail.com
> > Date: Fri, 29 Jan 2016 16:32:57 +
> > Subject: Re: [Tutor] lc_ctype and re.LOCALE
> > To: sjeik_ap...@hotmail.com
> > CC: tutor@python.org
> > 
> > On 28 January 2016 at 20:23, Albert-Jan Roskam <sjeik_ap...@hotmail.com> 
> > wrote:
> > >
> > > Out of curiosity, I wrote the throw-away script below to find a character 
> > > that is classified (--> LC_CTYPE) as digit in one locale, but not in 
> > > another.
> > > I ran it with 5000 locale combinations in Python 2 but did not find any 
> > > (somebody shut down my computer!). I just modified the code so it also
> > > runs in Python 3. Is this the correct way to find such locale-dependent 
> > > regex matches?
> > 
> > Eryk already gave you a better explanation of the locale stuff than I
> > could but I have a separate comment about the algorithmic performance
> > of your code (since you mentioned that it took a long time).
> > 
> > You're looping over all pairs of locales:
> > 
> > ...
> > > for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 
> > > 2), >
> > ...
> > > for i in xrange(sys.maxunicode + 1):   # 1114111
> > > s = unichr(i)  #.encode("utf8")
> > > try:
> > > locale.setlocale(locale.LC_CTYPE, locale1)
> > > m1 = bool(regex.match(s))
> > > locale.setlocale(locale.LC_CTYPE, locale2)
> > > m2 = bool(regex.match(s))
> > > if m1 ^ m2:  # m1 != m2
> > 
> > Suppose there are N locales and M is sys.maxunicode. The number of
> > pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
> > loop over M characters so the innermost loop body is repeated
> > something like M*N**2 times.
> > 
> > Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
> > your code above. We can swap the loops around so that the outer loop
> > is over unicode characters. Then the inner loop can be over the
> > locales but we only loop over all N locales once rather than over all
> > N**2 pairs of locales. This looks like this:
> > 
> > for c in unicode_chacters:
> > matched = f(locales[0], c) # Check the first locale
> > for locale in locales:
> > assert all(f(locale, c) == matched for locale in locales)
> > 
> > This way you call f(locale, c) M*N times which if N is not small
> > should be a lot faster than M*N**2 times.
> 
> Hi Oscar,
> 

Oh, it seems NoScript or something messed up Hotmail. Here is what I intended 
to send:

I blindly followed a code tuning tip I once read about in Code Complete 
(McConnel 2004; page 623 [1]):
 " Putting the Busiest Loop on the Inside
 
 When you have nested loops, think about which loop you want on the outside and
 which you want on the inside. Following is an example of a nested loop that 
can be
 improved:
 ...
 The key to improving the loop is that the outer loop executes much more often 
than the
 inner loop. Each time the loop executes, it has to initialize the loop index, 
increment it
 on each pass through the loop, and check it after each pass" 
 
 [1] 
https://khmerbamboo.files.wordpress.com/2014/09/code-complete-2nd-edition-v413hav.pdf
 
 Your advice makes perfect sense, though. So McConnel's code tuning tip may be 
a rule-of-thumb, with exceptions, right?




> 
> Thanks!
> 
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html

[Albert-Jan Roskam]

from sqlalchemy import *

db = create_engine('sqlite:///joindemo.db')

db.echo = True

metadata = BoundMetaData(db)

users = Table('users', metadata, autoload=True)
emails = Table('emails', metadata, autoload=True)

# These are the empty classes that will become our data classes
class User(object):
pass
class Email(object):
pass

usermapper = mapper(User, users)
emailmapper = mapper(Email, emails)

session = create_session()

mary = session.query(User).selectfirst(users.c.name=='Mary')
mary.age += 1

session.flush()

fred = User()
fred.name = 'Fred'
fred.age = 37

print "About to flush() without a save()..."
session.flush()  # Will *not* save Fred's data yet

session.save(fred)
print "Just called save(). Now flush() will actually do something."
session.flush()  # Now Fred's data will be saved

session.delete(fred)
session.flush()   
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

> Date: Wed, 27 Jan 2016 02:24:35 +1100
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
>
> On Sun, Jan 24, 2016 at 07:47:47PM +, Albert-Jan Roskam wrote:
>
>>> You appear to be confusing ordered and sorted.
>>
>> You are correct. Is there a difference in the way those terms are
>> used colloquially vs. in the field of Computer Science (Note: English
>> is not my mother tongue)?
>
> In ordinary English, "ordered" and "sorted" often are used to mean the
> same thing. People do often use sorted and ordered as interchangeable,
> but the definitions are slightly different:
>
>
>
> ordered \ordered\ adj.
> 1. having or evincing a systematic arrangement; especially,
> having elements succeeding in order according to rule; as,
> an ordered sequence; an ordered pair. Opposite of
> disordered or unordered. [Narrower terms:
> abecedarian, alphabetical; {consecutive, sequent,
> sequential, serial, successive ]
> [WordNet 1.5 +PJC]
>
> 2. arranged in order.
>
> Sort \Sort\, v. t. [imp. & p. p. Sorted; p. pr. & vb. n.
> Sorting.]
> 1. To separate, and place in distinct classes or divisions,
> as things having different qualities; as, to sort cloths
> according to their colors; to sort wool or thread
> according to its fineness.
> [1913 Webster]
>
>
> The way I would put it is that "sorted" means the items are ordered
> according to some specific rule or property of the items themselves,
> e.g. to sort your clothes by colour. "Ordered" is more general: it just
> means to have some order, which may be according to a rule or property,
> or it may be in whatever sequence the items happen to have.
>
> Books on a shelf have some order, the order that they appear when you
> read them from left to right, regardless of whether they are sorted by
> author, title, height, colour or at random. Books jumbled up in a bag
> have no order.
>
> Ordinary dicts are like books in a bag. You reach in and grab whatever
> book happens to come to hand first. OrderedDicts are like books on a
> shelf: you can systematically touch each book in order starting from the
> left, and new books are always added to the right.

Thank you! That distinction is indeed quite subtle. With "ordered" I tend to 
think of "ordinal measurement level" or something like that. E.g., sort a list 
of heights, calculate the median, ntiles etc. But your description makes it a 
whole lot clearer. Sometimes analogies work better!

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

 
> To: tutor@python.org
> From: alan.ga...@btinternet.com
> Date: Fri, 22 Jan 2016 00:12:18 +
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On 22/01/16 00:00, Steven D'Aprano wrote:
> 
> > Also, you have a problem -- what happens if the incomplete postcode is 
> > missing the first digit? (I suppose for that case, you just have to do a 
> > slow linear search.) 
If the postcode letters are missing, I look up the insertion point for the 
postcode digits. I then indeed use a linear search to loop through that postal 
code 4-digits area. Then I use the apartment number to filter out impossible 
street(s). Often postcodes for odd and even house numbers differ. So if, for 
instance, I have postcode digits 1234, no postcode letters, and apartment 
number 312, I can ignore all odd number ranges, and all ranges that do not 
include 312 (many streets have just a few dozen numbers). I then end up with a 
bunch of options, which I call candidates. Finally, I use 
difflib.SequenceMatcher to try and select a record of which the "dirty" street 
name, most closely resembles the standard street name in the list of 
candidates. I did not consider what to do when one postcode letter is missing, 
or when letters are transposed. I expect (and hope!) that the street/city 
cannot be derived and that this record needs to be reviewed manually. And in 
case you wondered: yes, it
  sucks that all these corrections need to be done "after the fact". A good 
reason to learn how to use Django or Flask to create a data entry screen and 
start with crisp clean data :-)
> Which is why a different solution may be better suited.
> What about an in-memory SQLite table? Then you can use
> a LIKE based select and let the library do the hard work.
Ouch, I considered marshall and pickle to cache the data, because running the 
query on MS SQL takes more time than I like. But I am embarrassed to say I did 
not even consider SQLite. I will reconsider this. Thank you! 
> Just a thought.
> 
> -- 
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

 
> To: tutor@python.org
> From: breamore...@yahoo.co.uk
> Date: Thu, 21 Jan 2016 21:02:03 +
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On 20/01/2016 13:33, Albert-Jan Roskam wrote:
> > Hi,
> >
> > Like the subject says: Why is an OrderedDict not sliceable? (From the 
> > collections library). Was that an intentional omission, or a mistake? [1]
> 
> Plenty of answers on this all ready, but...
> 
> >
> > Background: I do not use OrderedDict very often, but I thought I could use 
> > it to look up street and city names using postcodes ([0-9]{4} [a-z]{2} 
> > format). I needed it to be ordered because I also wanted to be able to use 
> > bisect, which is needed when the postcode letters are missing. In short: a 
> > fast dict lookup for complete postcodes and less fast bisect lookup for in 
> > complete postcodes.
> >
> 
> You appear to be confusing ordered and sorted.   You are correct. Is there a 
> difference in the way those terms are used colloquially vs. in the field of 
> Computer Science (Note: English is not my mother tongue)? Anyway, this page 
> seems to suggest that "Ordered" means "gets messed up upon insertion, 
> deletion, update: 
> http://stackoverflow.com/questions/1084146/what-is-the-difference-between-an-ordered-and-a-sorted-collection
>There is no way that you 
> can use bisect on an OrderedDict unless it is sorted in the first place.
 I fetch the data from a MS SQL Server with a query that goes something like 
SELECT DISTINCT pc_digits, pc_letters, house_number_from, house_number_to, 
street, city WHERE ... ORDER BY pc_digits, pc_letters.  
> > [1] http://stackoverflow.com/questions/30975339/slicing-a-python-ordereddict
> >
> > Thanks!
> >
> > Albert-Jan
> > 
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

> Date: Fri, 22 Jan 2016 11:00:00 +1100
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> Further thoughts on your question...
> 
> 
> On Wed, Jan 20, 2016 at 01:33:17PM +, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > Like the subject says: Why is an OrderedDict not sliceable? (From the 
> > collections library). Was that an intentional omission, or a mistake? 
> > [1]
> > 
> > Background: I do not use OrderedDict very often, but I thought I could 
> > use it to look up street and city names using postcodes ([0-9]{4} 
> > [a-z]{2} format). I needed it to be ordered because I also wanted to 
> > be able to use bisect, which is needed when the postcode letters are 
> > missing. In short: a fast dict lookup for complete postcodes and less 
> > fast bisect lookup for in complete postcodes.
> 
> I'm not sure I understand your use-case here.
> 
> You have postcodes that look like this:
> 
> "1234az"
> 
> Correct? Why do you want them *ordered*? 
> 
> I think you are confusing OrderedDict for a "Sorted Dict". OrderedDict 
> doesn't keep the keys in sorted order, it keeps them in the order that 
> they were inserted. So unless you are super-careful to insert the 
> postcodes in sorted order, the order of them in the dict will be 
> whatever order you insert them:

You are right. See also my reply to Mark Lawrence, who made a similar remark.

> py> from collections import OrderedDict
> py> d = OrderedDict()
> py> d['1234az'] = "1 Smith Street"
> py> d['zz'] = "991203 Short Street"
> py> d['3456mx'] = "24 Hour Lane"
> py> for key in d:
> ... print(key, d[key])
> ...
> 1234az 1 Smith Street
> zz 991203 Short Street
> 3456mx 24 Hour Lane
> 
> 
> So even if OrderedDict supported slicing, that would not do what you 
> think it does.


Oscar Benjamin's link about collections.OrderedDict.__eq__ (or rather, the fact 
that it's not reimplemented) scared the heck outta me :-)
Maybe this class should be avoided altogether?
 
> Also, you have a problem -- what happens if the incomplete postcode is 
> missing the first digit? (I suppose for that case, you just have to do a 
> slow linear search.) What about transposed digits or other errors? I'm 
> glad I don't have to solve that problem!
> 
> 
> Anyway, I suggest doing something like this:
> 
> (1) Keep the known postcodes in a regular dict, not an ordered dict.
> 
> (2) Once you have built the dict, then copy the keys and sort them:
> 
> postcodes = {
> '1234az': "1 Smith Street",
> 'zz': "991203 Short Street",
> '3456mx': "24 Hour Lane",
> }
> 
> array = sorted(postcodes.keys())
> 
> 
> (3) Each time you add a new postcode to the dict, use bisect to add it 
> to the array as well. To ensure you never forget, use a helper function:

But I only *read* from the postcode table. I never insert any postcodes, nor do 
I delete any (ok, a few times a year I load new definitions from the database, 
because new houses, with new postcodes, will have been built).  See also my 
mail to Alan (today).

> def insert(postcode, entry):
> if postcode in postcodes:
> # deal with duplicate/existing key
> ...
> else:
> postcodes[postcode] = entry
> bisect.insort(array, postcode)
> 
> 
> Same for deleting.
> 
> 
> 
> -- 
> Steve
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

> From: oscar.j.benja...@gmail.com
> Date: Thu, 21 Jan 2016 11:02:40 +
> To: ben+pyt...@benfinney.id.au
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> CC: tutor@python.org
> 
> On 21 January 2016 at 09:19, Ben Finney <ben+pyt...@benfinney.id.au> wrote:
> > Albert-Jan Roskam <sjeik_ap...@hotmail.com> writes:
> >
> >> Why is an OrderedDict not sliceable?
> >
> > Because slicing implies index access. The built-in ‘dict’ and
> > ‘collections.OrderedDict’ both do not support indexed access::
> 
> According to a narrow definition of indexed access. I would say that
> d[k] is index access even if d is a dict and k a key.
> 
> Albert-Jan I guess what you want is this:
> 
> from collections import OrderedDict
> 
> class SliceableOrderedDict(OrderedDict):
> def __getitem__(self, sl):
> if isinstance(sl, slice):
> keys = list(self)
> keyslice = slice(
> None if sl.start is None else keys.index(sl.start),
> None if sl.stop is None else keys.index(sl.stop),
> sl.step
> )
> newitems = ((k, self[k]) for k in keys[keyslice])
> return SliceableOrderedDict(newitems)
> else:
> return super().__getitem__(sl)
 
 
That looks interesting. I will check this out tomorrow at work. If I read it 
correctly this is indeed exactly what I meant. Thank you!!

 
> I guess that the authors of OrderedDict just didn't really consider
> this to be very useful. Apart from having ordered iteration
> OrderedDict is not really that deeply thought out. There's a thread on
> python-ideas about the inconsistent behaviour of the keys and values
> views and equality comparison of OrderedDicts on python-ideas at the
> moment:
> 
> https://mail.python.org/pipermail/python-ideas/2015-December/037472.html

As I said in a prior email: That is SCARY! Should I just avoid OrderedDict like 
the plague?
 

> --
> Oscar
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

> To: tutor@python.org
> From: ben+pyt...@benfinney.id.au
> Date: Fri, 22 Jan 2016 04:12:08 +1100
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> Ben Finney  writes:
> 
> > Oscar Benjamin  writes:
> >
> > > According to a narrow definition of indexed access. I would say that
> > > d[k] is index access even if d is a dict and k a key.
> 
> The sense of “index” implied is used consistently throughout Python
>  to refer to the integer
> ordinal position in a sequence.

I appear to have confused the terms "sorted" and "ordered" (see the email I 
just sent to Mark Lawrence).  My OrderedDict was sorted on its keys, because I 
defined the dict using the result of an SQL query that ended with ORDER BY 
. So in that case I needed a kind of "chameleon" 
datatype: both a mapping and an indexing type [1]
[1] https://docs.python.org/2/reference/datamodel.html#object.__getitem__ 
 

> It is not compatible with key access into a mapping.
> 
> > An index implies the ordinal position in a sequence. In a mapping, the
> > key is *not* referring to the position in a sequence, so is not a key.
> 
> “the key … is not an index”, I mean.
> 
> > So accessing an item in a mapping by key is not indexed access.
> 
> -- 
>  \ “Facts do not cease to exist because they are ignored.” —Aldous |
>   `\Huxley |
> _o__)  |
> Ben Finney
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Change datatype for specific columns in an 2D array & computing the mean

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Sun, 24 Jan 2016 11:22:19 +0100
> Subject: Re: [Tutor] Change datatype for specific columns in an 2D array & 
> computing the mean

 
> How do you want to convert the second and third column to int? Are A4 and B2 
> hex numbers? Then try
> 
> $ cat esawi.csv 
> 19,A4,B2,2
> 19,A5,B1,12
> $ python3
> Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
> [GCC 4.8.4] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy
> >>> def fromhex(s):
> ... return int(s, 16)
> ... 
> >>> numpy.genfromtxt("esawi.csv", delimiter=",", converters={1:fromhex, 
> 2:fromhex})
> array([(19.0, 164, 178, 2.0), (19.0, 165, 177, 12.0)], 
>   dtype=[('f0', '

Re: [Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

> Date: Thu, 21 Jan 2016 12:00:29 +1100
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On Wed, Jan 20, 2016 at 01:33:17PM +, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > Like the subject says: Why is an OrderedDict not sliceable? (From the 
> > collections library). Was that an intentional omission, or a mistake? 
> > [1]
> 
> Because slicing a dict makes no sense. A dict is a mapping, not a 
> sequence.

For a regular dict: yes, I agree that doesn't make sense, because a regular 
dict is unordered.
A collections.OrderedDict on the other hand..

> d = OrderedDict()
> d["cow"] = "a"
> d["fox"] = "b"
> d["ape"] = "c"
> d["dog"] = "d"
> d["bee"] = "e"
> 
> I can do a dict lookup on a key:
> 
> d["cow"]
> 
> What would a slice even mean? d[1:4] but 1, 2, 3 are not keys of the 
> dict.

Indexing would create ambiguity, e.g d[1] might mean
* return the value associated with key 1
* return the first value

But that's not true for slices, because they cannot be dictionary keys anyway:
>>> {slice(1, 2): None}
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unhashable type

With islice it is easy to do it anyway, but I still find it strange that 
OrderedDict.__getitem__ does not do that already (sorry!)
>>> from itertools import islice
>>> from collections import OrderedDict
>>> od = OrderedDict({"a": 1, "b": 2})
>>> list(islice(od, 1, 2))
['b']


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Why is an OrderedDict not sliceable?

[Albert-Jan Roskam]

Hi,

Like the subject says: Why is an OrderedDict not sliceable? (From the 
collections library). Was that an intentional omission, or a mistake? [1]

Background: I do not use OrderedDict very often, but I thought I could use it 
to look up street and city names using postcodes ([0-9]{4} [a-z]{2} format). I 
needed it to be ordered because I also wanted to be able to use bisect, which 
is needed when the postcode letters are missing. In short: a fast dict lookup 
for complete postcodes and less fast bisect lookup for in complete postcodes.

[1] http://stackoverflow.com/questions/30975339/slicing-a-python-ordereddict

Thanks!

Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] str.strip strange result...?

[Albert-Jan Roskam]

 
> From: d...@hashcollision.org
> Date: Mon, 18 Jan 2016 00:29:41 -0800
> Subject: Re: [Tutor] str.strip strange result...?
> To: sjeik_ap...@hotmail.com
> CC: __pete...@web.de; tutor@python.org
> 
> > Not sure which one is faster, but in this case I actually find a regex more 
> > readable (!):
> >>>> re.sub(r"_1$", "", "V01_1")
> > 'V01'
> 
> 
> Hi Albert-Jan Roskam,
> 
> Here's a minor counterpoint to using regexes here: they're sometimes a
> bit too powerful. Yes, certainly: https://xkcd.com/208/ :-)
> 
> In this situation, a regex approach might be troublesome if parts of
> the suffix can be interpreted as regular expression meta-characters.
> For example, if we were trying to strip out a suffix like
> 
> ".exe"
> 
> then if we were to try to do this with regexes, we might forget that
> "." is a meta-character that acts as a wildcard for any single
> character.  Therefore, a regexp-based solution for the general
> suffix-removal problem is complicated because we'd need consider
> escaping certain characters.  Not that this would be hard, but that
> it's yet another detail we have to keep in our heads.
Agreed. Apart from backslashes, re.escape could also be really handy. But 
regexes are scary and intimidating for those who have never seen them before.
> Another disadvantage is that, if the suffix is dynamically determined,
> then there's an additional cost in "compiling" the regular expression:
> building the pattern-matching machinery doesn't come for free.
re.compile helps, though it always bothers me that I "need" to put them as a 
global in the top-level of my module (along with collections.namedtuple).Unless 
I use a class, where __init__ is a good place.
 
> For those reasons, I think the regexp approach here is a little bit of
> overkill.  This kind of tradeoff is the sort of thing that reference
> documentation will probably not mention.   That's why this list is
> here, to share the experience of using these systems with other
> beginners and learners.  Regexes are still very useful: just be aware
> that they have sharp edges.
 One other downside is that it takes only a couple of days/beers before one 
forgets the regex. But re.VERBOSE and named groups help a LOT.
 
> Good luck to you!
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] me, my arm, my availability ...

[Albert-Jan Roskam]

> To: tutor@python.org
> From: cmgcom...@gmail.com
> Date: Thu, 14 Jan 2016 09:37:02 +0530
> Subject: Re: [Tutor] me, my arm, my availability ...
> 
> take care, get well soon,
> 
> regards
> 
> CMG
> 
> On Thursday 14 January 2016 02:17 AM, Laura Creighton wrote:
> > I fell recently.  Ought to be nothing, but a small chip of bone, either an
> > existing one or one I just made is nicely wedged in the joint taking away
> > a whole lot of the ability of my arm to rotate in the elbow joint.  Or
> > hold my arm in a position that is usual for typing.  Plus,  now that the
> > sprain/swelling is more or less over, the pain, unfortunately is not.
> >
> > The real downside is that my typing speed is down from 135-140 wpm
> > to 5-10 wmp.  At this rate, just getting my usual work done takes
> > overtime.
> >
> > Seems like surgery is needed to fix this.
> >
> > So I wanted you all to know, no, I haven't forgotten you and no haven't
> > stopped caring.  I have just stopped being as __capable__ if you know
> > what I mean.
> >
> > Please take care of yourselves and each other.  I will often be reading
> > even if typing is more than I can do right now.

awww, that sucks. Sorry to hear that. My daughter had something similar last 
year. Barely visible on x-rays, but very painful and it took weeks to heal.
Take care!

Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] str.strip strange result...?

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Fri, 15 Jan 2016 17:37:25 +0100
> Subject: Re: [Tutor] str.strip strange result...?
> 
> Jignesh Sutar wrote:
> 
> > #python2.7
> > 
>  s="V01_1"
>  s.strip("_1")
> > 'V0'
> > 
> > 
> > Wouldn't you expect the result to be "V01" ?
> 
> str.strip() doesn't strip off a suffix or prefix; its first argument is 
> interpreted as a character set, i. e. as long as s ends/starts with any of 
> the characters "_" or "1", remove that.
> 
> If you want to remove a suffix you have to write
> 
> if suffix and s.endswith(suffix):
> s = s[:-len(suffix)]

Not sure which one is faster, but in this case I actually find a regex more 
readable (!):
>>> re.sub(r"_1$", "", "V01_1")
'V01'

Without the $ sign may also do the trick, but:
>>> re.sub(r"_1", "", "V01_1_1")
'V01'
>>> re.sub(r"_1$", "", "V01_1_1")
'V01_1'


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question about the memory manager

[Albert-Jan Roskam]

> From: eryk...@gmail.com
> Date: Thu, 14 Jan 2016 04:42:57 -0600
> Subject: Re: [Tutor] Question about the memory manager
> To: tutor@python.org
> CC: sjeik_ap...@hotmail.com
> 
> On Thu, Jan 14, 2016 at 3:03 AM, Albert-Jan Roskam
> <sjeik_ap...@hotmail.com> wrote:
> >
> > These two pages are quite nice. The author says the memory used by small 
> > objects is
> > never returned to the OS, which may be problematic for long running 
> > processes.
> 
> The article by Evan Jones discusses a patch to enable releasing unused
> arenas (i.e. "how the problem was fixed"). Starting with 2.5, unused
> arenas do get released back to the heap. Here's the diff in which Tim
> Peters merged in a "heavily altered derivative" of Evan's patch [1].
> 
> Also, 2.7 and 3.3 bypass C malloc/free and the process heap to instead
> use mmap/munmap on POSIX when available. This avoids the heap
> high-water mark problem. Similarly, 3.4 switched to using
> VirtualAlloc/VirtualFree on Windows. 3.4 also introduced the
> PyObjectArenaAllocator and associated C API functions [2] to allow
> modifying the default allocators.
> 
> [1]: https://hg.python.org/cpython/diff/685849bd905c/Objects/obmalloc.c
> [2]: 
> https://docs.python.org/3/c-api/memory.html#customize-pyobject-arena-allocator

Hi Eryk,

Thanks a lot for the info and the links. This is truly interesting to read 
about! Glad to know that the high-water mark problem is no longer relevant 
anymore in recent Python versions. Also, thank you for your suggestion about 
psutils (and ctypes).

Best wishes,
Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question about the memory manager

[Albert-Jan Roskam]

D

> From: sjeik_ap...@hotmail.com
> To: tim.pet...@gmail.com
> Date: Wed, 13 Jan 2016 08:11:11 +
> Subject: Re: [Tutor] Question about the memory manager
> CC: tutor@python.org
> 
> > From: tim.pet...@gmail.com
> > Date: Sun, 10 Jan 2016 10:54:10 -0600
> > Subject: Re: [Tutor] Question about the memory manager
> > To: sjeik_ap...@hotmail.com
> > CC: tutor@python.org
> > 
> > [Albert-Jan Roskam <sjeik_ap...@hotmail.com>]
> > > I just found a neat trick to free up an emergency stash of memory in
> > > a funtion that overrides sys.excepthook. The rationale is that all
> > > exceptions, including MemoryErrors will be logged.
> > > The code is below. My question: is that memory *guaranteed* to be
> > > freed right after the 'del' statement? Or should one call gc.collect to
> > > be really sure?
> > >
> > > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > > def handle_exception(e):
> > > global rainydayfund
> > > del rainydayfund
> > > ... etc, etc ...
> > > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> > 
> > This works fine in all versions of CPython (the C implementation of
> > Python distributed by python.org) to date.  That's because:
> > 
> > 1. All versions of CPython rely primarily on reference counting (`gc`
> > is only needed to reclaim garbage containing reference cycles).  An
> > object is released immediately when its reference count falls to 0.
> > 
> > 2. There is only one reference to the big list there (via the global
> > `raindydayfund`), so the memory becomes garbage immediately upon
> > executing the `del`.
> > 
> > 3. Similarly, that giant list holds the only references to the masses
> > of distinct empty lists it contains, so they also become garbage
> > immediately upon the giant list becoming garbage.
> > 
> > 4. CPython doesn't happen to stick garbage lists in, e.g., some
> > internal free list reusable only for new list objects - it actually
> > releases the memory for garbage lists.  Kinda ;-)
> > 
> > #2 and #3 are necessarily true.  #1 is true in CPython, but not in all
> > implementations of Python.
> > 
> > #4 is where things _might_ change even in CPython, but it's very
> > unlikely to change.  As is, it would take a small book to flesh out
> > what "Kinda ;-)" means, exactly.  Memory management is complex, with
> > many layers, involving many details.
> > 
> > If you can live with all that, I'd suggest a more straightforward way
> > of setting it up, like:
> > 
> > rainydayfund  = b"x" * N
> > 
> > where `N` is the number of bytes you want to reserve.  That is, create
> > a giant bytestring containing the number of "emergency bytes" you
> > need.  If N is large enough, that will avoid CPython's "small object
> > allocator" and CPython's "arena allocator", getting the memory
> > directly from (and returning the memory directly to) the OS.  The
> > fewer layers that get involved, the fewer layers that _may_ surprise
> > you by changing behavior in the future.
> 
> Hi Tim,
> 
> Thank you! Can you recommend a document or website that CPython's memory 
> manager?
> Might be interesting and perhaps useful to know a bit more about the details. 
> Perhaps this knowledge might sometimes help writing faster code?
> 

These two pages are quite nice. The author says the memory used by small 
objects is never returned to the OS, which may be problematic for long running 
processes. It appears that it is better to have a few big objects rather than 
many small ones, because memory is more likely to become fragmented with many 
deleted small objects (and there's no such thing as gc.defrag)
http://www.evanjones.ca/memoryallocator/
http://deeplearning.net/software/theano/tutorial/python-memory-management.html


> Best wishes,
> Albert-Jan
> 
> PS:
> 
> albertjan@debian:~$ python -c "import this" 
> The Zen of Python, by Tim Peters
> 
> ...
> ...
> There should be one-- and preferably only one --obvious way to do it.
> Although that way may not be obvious at first unless you're Dutch.
> 
> --> Nope not even then. Perhaps if my name were Guido? :-)
> 
> 
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question about the memory manager

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Sun, 10 Jan 2016 18:29:06 +0100
> Subject: Re: [Tutor] Question about the memory manager
> 
> Albert-Jan Roskam wrote:
> 
> > Hi,
> > 
> > I just found a neat trick to free up an emergency stash of memory in a
> > funtion that overrides sys.excepthook. The rationale is that all
> > exceptions, including MemoryErrors will be logged. The code is below. My
> > question: is that memory *guaranteed* to be freed right after the 'del'
> > statement? Or should one call gc.collect to be really sure?
> > 
> > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > def handle_exception(e):
> > global rainydayfund
> > del rainydayfund
> > ... etc, etc ...
> > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> 
> I must admit that this looks rather strange to me, but I don't see any 
> problem why it wouldn't work. gc.collect() only helps with circular 
> dependencies, and there aren't any in your "rainy day fund".
> 
> Regarding the use of sys.excepthook I'd rather (SO voting be damned!) take 
> the obvious approach as suggested by Vinay Sajip, i. e.
> 
> try:
> main()
> except:
> # do what you have to do
> 
> instead of rewriting the hook. If potential memory resource hogs are 
> confined within main() with no references from any module namespace you 
> should have enough memory availaible to run the except suite without the 
> need for a rainy day fund. If you know about a specific global name that 
> references a consumer of a lot of memory, an in-memory db, say, then why not 
> handle the memory error there or at least use it in lieu of a dedicated 
> dummy? I. e.
> 
> in_memory_db = None
> try:
> try:
> main()
> finally:
> del in_memory_db
> except:
> # do what you have to do

Hi all,

Thanks for your replies. Initially my main concern was to also log 
fatal/unhandled exceptions. The database MemoryError was also I was also an 
issue I was just facing, so Martelli's approach seemed nice. Meanwhile I found 
out I did not only delete "TOP 100" (which I used to debug my code) in my 
SELECT query, but also DISTINCT. After I corrected this MemoryErrors are no 
longer an issue with this function. :-) Still, I always wonder how close I am 
to a MemoryError (I am on a tiny Win 7 32 VM, with Python 2.7). I sometimes 
check the Task Manager to see how much RAM is in use. Is there a Python way to 
do something similar?

Best wishes,
Albert-Jan


  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Question about the memory manager

[Albert-Jan Roskam]

Hi,

I just found a neat trick to free up an emergency stash of memory in a funtion 
that overrides sys.excepthook. The rationale is that all exceptions, including 
MemoryErrors will be logged.
The code is below. My question: is that memory *guaranteed* to be freed right 
after the 'del' statement? Or should one call gc.collect to be really sure?

rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
def handle_exception(e):
global rainydayfund
del rainydayfund
... etc, etc ...
http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way

Thanks!

Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] a class that may not be instantiated

[Albert-Jan Roskam]

 Hi all,
 
Thanks a lot for your replies!
 
> Date: Wed, 25 Nov 2015 05:19:57 +1100
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] a class that may not be instantiated
> 
> On Tue, Nov 24, 2015 at 03:36:21PM +, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > I have two classes with a number of methods that are the same, so I 
> > want to define a super class that holds these methods.
> 
> Sounds like a misuse of classes to me. DRY is not a good reason to make 
> two otherwise unrelated classes subclasses of an otherwise pointless 
> parent.
> 
> Better solutions include:
> 
> - Factor out the common code to a module-level function, and have the 
> classes call that function from their methods:
 > - Use a Mixin class to hold the shared methods, and "mix them in" to 
the 
> two other classes as needed.
 I think I like this option best. There is as little "visual clutter" in 
the child classes as possible, but the user does not get confusing info about 
parent-child relationships.When the methods are converted to functions, one 
still needs to implement a (very short) method in the child class.But the term 
"Mixin" is just a convention, right? (Similar to a single leading underscore) 
It's not that the class name gets mangled whenever it contains the string 
"Mixin" (similar to name mangling with two leading underscores). > But, let's 
say you still want to proceed with your first plan:
> 
> 
> > But the super 
> > class (below called _Generic) should not be instantiated, because it 
> > serves no purpose other than the DRY principle. I raise a 
> > NotImplementedError in case if somebody dares to instantiate _Generic.
> 
> Sounds reasonable, although you should be kind to your subclasses:
> 
> class _Generic(object):
> def __init__(self, *args, **kwargs):
> if type(self) is _Generic:
> raise NotImplementedError('abstract base class cannot be 
> instantiated')
> 
> 
> Now your concrete subclasses aren't forced to override __init__ if they 
> don't need to. That "if type"check is a very nice addition indeed. That was 
> also my objection against the use of an abstract method with abc, as Peter 
> mentioned:with abc the __init__ *must* be implemented in the concrete class. 
> Not usually a big problem, but still...  
regards,Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] a class that may not be instantiated

[Albert-Jan Roskam]

Hi,

I have two classes with a number of methods that are the same, so I want to 
define a super class that holds these methods. But the super class (below 
called _Generic) should not be instantiated, because it serves no purpose other 
than the DRY principle. I raise a NotImplementedError in case if somebody dares 
to instantiate _Generic. Below, only one common method is defined in_Generic, 
namely __repr__ (the other ones are __enter__ and __exit__, maybe more, if you 
must know). At first I thought I'd need the abc module for this, but this seems 
to do the trick. I do not want to enforce concrete implementations of abstract 
methods, which is the goal of abc. Is this the way to do this, or this this 
quirky code?

import inspect

class _Generic(object):

    def __init__(self, *args, **kwargs):
    raise NotImplementedError

    def __repr__(self):
    fmt = []
    for arg in inspect.getargspec(self.__init__).args[1:]:
    value = getattr(self, arg)
    sr = "%r" if isinstance(value, basestring) else "%s"
    fmt.append(("%s=" + sr) % (arg, value))
    return self.__class__.__name__ + "(" + ", ".join(fmt) + ")" 
    
class Concrete(Generic):

    def __init__(self, x=42, y='a'):
    self.x = x
    self.y = y
    
class OneMore(Generic):

    def __init__(self, w=555, z='foo'):
    self.w = w
    self.z = z

    
c = Concrete()
print repr(c)

a = _Generic(666)   #  NotImplementedError


Thank you!


Albert-Jan
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Missing posts

[Albert-Jan Roskam]

(Sorry for top-posting)

Could it be related to this? 
https://www.emailonacid.com/blog/article/industry-news/could_yahoo_and_aols_dmarc_policies_destroy_your_deliverability

Albert-Jan

> To: tutor@python.org
> From: alan.ga...@btinternet.com
> Date: Sun, 15 Nov 2015 00:08:37 +
> Subject: [Tutor] Missing posts
> 
> Hi Folks,
> 
> I've approved a couple of messages in the last 24 hours that have
> not made it onto the list. I'm not sure what's happening but, if
> you posted recently and it has not shown up, please feel free
> to post again.
> 
> Meantime, I'm going to try to find out what has happened.
> 
> 
> -- 
> Alan G
> Moderator
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python ASTM Implementation

[Albert-Jan Roskam]

> To: mokshavi...@gmail.com
> CC: alan.ga...@btinternet.com; tutor@python.org; sjeik_ap...@hotmail.com; 
> l...@openend.se
> From: l...@openend.se
> Subject: Re: [Tutor] Python ASTM Implementation
> Date: Fri, 13 Nov 2015 12:04:27 +0100
> 
> In a message of Fri, 13 Nov 2015 10:07:49 +0530, "Br. Sayan" writes:
> 
> >Maybe it's setuptools that is too old? Or perhaps pip?
> >> sudo apt-get update
> >> sudo apt-get install --only-upgrade python-setuptools python-pip
> >>
> >> I believe it's safer to install Python packages via apt-get if you are
> >> using the Linux internal Python, rather than using pip.
> >>
> >
> >I have not installed through pip. I just gave the setup script 777
> >permission and run it with ./setup.py . How do I do it with apt-get? I
> >don't think it's in repository. Pls let me know.
> 
> It is not in the repository (at least in mine in debian unstable.)
> Here is how you find such things out:


I was not suggesting to install astm via apt-get, just setuptools (and while 
we' re at it, pip as well).
After you've done that, you could try to install astm the way you intended to 
(apparently using setup.py install, which tries to import distutils or 
setuptools).

 
> $ apt-cache search astm
> python-django-model-utils - Django model mixins and utilities
> python3-django-model-utils - Django model mixins and utilities
> libcache-fastmmap-perl - Perl module providing a mmap'ed cache
> libcatalyst-plugin-cache-store-fastmmap-perl - (deprecated) FastMmap cache 
> store plugin for Catalyst::Plugin::Cache
> libcatalyst-plugin-session-store-fastmmap-perl - Catalyst session storage 
> plugin backed by Cache::FastMMap
> libcgi-application-plugin-tt-perl - plugin that adds Template Toolkit support 
> to CGI::Application
> r-cran-fastmatch - GNU R package for fast match replacement for repeated 
> look-ups
> 
> So, lots of packages have the string 'astm' in their names, and none
> of them are the one we are looking for.
> 
> Did you find
> http://python-astm.readthedocs.org/en/latest/
> 
> It's not much in the way of documentation, but a bit better than 
> nothing.
> 
> There are people recommending python-astm here.
> http://www.limsforum.com/  (just search for ASTM and you will get many
> hits).  I didn't find any tutorials, though.
> 
> I did find this, https://github.com/mpasternak/cobas-scraper
> which in no way is what you were looking for, but I thought, hmm, he
> might want one of these as well.
> 
> Laura
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about descriptors

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Fri, 13 Nov 2015 09:26:55 +0100
> Subject: Re: [Tutor] question about descriptors
> 
> Albert-Jan Roskam wrote:
> 
> >> __getattr__() is only invoked as a fallback when the normal attribute
> >> lookup fails:
> > 
> > 
> > Aha.. and "normal attributes" live in self.__dict__?
> 
> I meant "normal (attribute lookup)" rather than "(normal attribute) lookup".
> __getattr__() works the same (I think) when there is no __dict__: 
> 
> >>> class A(object):
> ... __slots__ = ["foo"]
> ... def __getattr__(self, name):
> ... print "looking for", name
> ... return 42
> ... 
> >>> a = A()
> >>> a.foo
> looking for foo
> 42
> >>> a.__dict__
> looking for __dict__
> 42
> >>> a.foo = "bar"
> >>> a.foo
> 'bar'

Thank you again for the explanation. Much appreciated. I had not even thought 
about __slots__ yet.

  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question about descriptors

[Albert-Jan Roskam]

> To: tutor@python.org
> From: __pete...@web.de
> Date: Wed, 11 Nov 2015 20:06:20 +0100
> Subject: Re: [Tutor] question about descriptors
> 
> Albert-Jan Roskam wrote:
> 
> >> From: st...@pearwood.info
> 
> >> Fortunately, Python has an mechanism for solving this problem:
> >> the `__getattr__` method and friends.
> >> 
> >> 
> >> class ColumnView(object):
> >> _data = {'a': [1, 2, 3, 4, 5, 6],
> >>  'b': [1, 2, 4, 8, 16, 32],
> >>  'c': [1, 10, 100, 1000, 1, 10],
> >>  }
> >> def __getattr__(self, name):
> >> if name in self._data:
> >> return self._data[name][:]
> >> else:
> >> raise AttributeError
> >> def __setattr__(self, name, value):
> >> if name in self._data:
> >> raise AttributeError('read-only attribute')
> >> super(ColumnView, self).__setattr__(name, value)
> >> def __delattr__(self, name):
> >> if name in self._data:
> >> raise AttributeError('read-only attribute')
> >> super(ColumnView, self).__delattr__(name)
> > 
> > That also seems very straightforward. Why does "if name in self._data:"
> > not cause a recursion? self._data calls __getattr__, which has self._data
> > in it, which...etc.
> 
> __getattr__() is only invoked as a fallback when the normal attribute lookup 
> fails:


Aha.. and "normal attributes" live in self.__dict__?


 
> >>> class A(object):
> ... def __getattr__(self, name):
> ... return self.data[name]
> ... 
> >>> a = A()
> >>> a.data = dict(foo="bar")
> >>> a.foo
> 'bar'
> >>> del a.data
> >>> import sys
> >>> sys.setrecursionlimit(10)
> >>> a.foo
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 3, in __getattr__
>   File "", line 3, in __getattr__
>   File "", line 3, in __getattr__
> RuntimeError: maximum recursion depth exceeded while calling a Python object
> 
> If you need to intercept every attribute lookup use __getattribute__():

Fantastic, thank you for the clear explanation. Do you happen to know whether 
the __getattr__ vs. __getattribute__ distinction was (a) a deliberate design 
decision or (b) a historic anomaly? If one considers the distinction between 
"normal attributes"  vs. "attributes of which the read/write/delete 
properties*) may be changed" , I'd say (a).
 
*) with files these are called "attributes", so one could call them attributes 
with attributes. :-)

> >>> class B(A):
> ... def __getattribute__(self, name):
> ... print "looking for", name
> ... return super(B, self).__getattribute__(name)
> ... 
> >>> b = B()
> >>> b.data = dict(foo="bar")
> >>> b.foo
> looking for foo
> looking for data
> 'bar'
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python ASTM Implementation

[Albert-Jan Roskam]

> Date: Thu, 12 Nov 2015 16:53:20 +0530
> From: mokshavi...@gmail.com
> To: tutor@python.org
> Subject: [Tutor] Python ASTM Implementation
> 
> Dear All,
> 
> We have a Lab Analyzer(Cobas C311), which we want to interface with Pyhton.
> As you might be knowing that clinical lab analyzers use ASTM protocol.
> Python has a an ASTM module  , but
> I need help in its implementation. In the first place, to my embarrassment
> I could not install it on Linux Mint 15 with Python 2.7.4.
> 
> The following error is coming :
> 
> 
> > /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown
> > distribution option: 'zip_safe'
> >   warnings.warn(msg)
> > /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown
> > distribution option: 'test_suite'
> >   warnings.warn(msg)
> > /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown
> > distribution option: 'install_requires'
> >   warnings.warn(msg)
> > usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
> >or: setup.py --help [cmd1 cmd2 ...]
> >or: setup.py --help-commands
> >or: setup.py cmd --help
> >
> > error: no commands supplied
> >
> 
> And next, cannot figure out how to start. Desperately need your help in
> starting off the coding. I have little experience in Python, but am
> confident that I can pick it up soon if I get help.
> 
> If you feel like, you may direct me to some other links where this type of
> codings have been shown with examples of some sort.

Hi,

Maybe it's setuptools that is too old? Or perhaps pip?
sudo apt-get update
sudo apt-get install --only-upgrade python-setuptools python-pip

I believe it's safer to install Python packages via apt-get if you are using 
the Linux internal Python, rather than using pip.

Albert-Jan




  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor