Re: [Tutor] Chunking list/array data?

[Peter Otten]

Sarah Hembree wrote:

> How do you chunk data? We came up with the below snippet. It works (with
> integer list data) for our needs, but it seems so clunky.
> 
> def _chunks(lst: list, size: int) -> list:
> return  [lst[x:x+size] for x in range(0, len(lst), size)]
> 
> What do you do? Also, what about doing this lazily so as to keep memory
> drag at a minimum?

If you don't mind filling up the last chunk with dummy values this will 
generate tuples on demand from an arbitrary iterable:

>>> from itertools import zip_longest
>>> def chunks(items, n):
... return zip_longest(*[iter(items)]*n)
... 
>>> chunked = chunks("abcdefgh", 3)
>>> next(chunked)
('a', 'b', 'c')
>>> list(chunked)
[('d', 'e', 'f'), ('g', 'h', None)]


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] which of these is more efficient?

[Peter Otten]

nathan tech wrote:

> Hi there,
> 
> So I am running over some coding ideas in my head for creating a map for
> a game.
> 
> This map would expand based on how far the user explores.
> 
> I figure there are two ways to do this:
> 
> 1: the list method:
> 
> map=[]
> 
> for x in range(3):
> 
>  temp=[]
> 
>  for y in range(3):
> 
>  temp.append(default_grid_format)
> 
>  map.append(temp)
> 
> 
> then when ever the user explores a square not on the current map, it
> would do this:
> 
> for x in range(len(map)):
> 
>  map[x].append(default_grid_format)
> 
> temp=[]
> 
> for x in range(len(map[0])):
> 
>  temp.append(default_grid_format)
> 
> map.append(temp)
> 
> Obviously, though, this creates a lot of data for squares that are still
> ultimately unexplored.
> 
> So here was my other idea:
> 
> 
> 2. the dictionary method:
> 
> map={}
> 
> for x in range(3):
> 
>  for y in range(3):
> 
>  key=str(x)+":"+str(y)
> 
>  map[key]=default_grid_format
> 
> 
> Then when user explores new square do:
> 
> key=str(player_x)+":"+str(player_y)
> 
> map[key]=default_grid_format
> 
> 
> Is this an efficient method compared to 1?
> 
> Is it, code wise, sound logic?
> 
> 
> 
> I guess I'm just looking for a second opinion from experienced peoples.
> 
> thanks everyone.

Forget about "efficiency", try to write clean code.
For the above samples this means

- no unused data
- no unnecessary stringification

If you base your code on a defaultdict cells spring into existence as 
needed:

$ cat tmp.py
from collections import defaultdict

def get_default_format():
   return "whatever"

def show_cell(x, y):
print(f"x = {x}, y = {y}, format = {grid_formats[x,y]!r}")

grid_formats = defaultdict(get_default_format)
grid_formats[42, 42] = "this cell is special"

player_location = 3, 4
show_cell(*player_location)
show_cell(42, 42)

$ python3.6 tmp.py
x = 3, y = 4, format = 'whatever'
x = 42, y = 42, format = 'this cell is special'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What is Tuple in the typing module?

[Peter Otten]

C W wrote:

> Hi everyone,
> 
> What exactly is Tuple in the typing module? What does it do?
> 
> This is the definition from its website.
> https://docs.python.org/3/library/typing.html
> "A type alias is defined by assigning the type to the alias"
> 
> I have no idea what that means.
> 
> Here's the example from the documentation:
> 
> from typing import Dict, Tuple, Sequence
> ConnectionOptions = Dict[str, str]Address = Tuple[str, int]Server =
> Tuple[Address, ConnectionOptions]
> def broadcast_message(message: str, servers: Sequence[Server]) -> None:
> ...
> # The static type checker will treat the previous type signature as#
> being exactly equivalent to this one.def broadcast_message(
> message: str,
> servers: Sequence[Tuple[Tuple[str, int], Dict[str, str]]]) ->
> None:
> ...
> 
> 
> I think this even more confusing. Can someone explain this in simple
> words? I don't have a intense computer science background.

After the line

ANSWER = 42

you can write

print("The answer is", ANSWER)

rather than

print("The answer is", 42)

Likewise, after

Location = Tuple[float, float, float]

"Location" has become an "alias" for a tuple of 3 floats, and you can write

def distance(a: Location, b: Location) --> float: ...

instead of

def distance(
a: Tuple[float, float, float],
b: Tuple[float, float, float]
) --> float: ...

Basically, use descriptive names for types rather than repeating their 
definition.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Search for Text in File

[Peter Otten]

Stephen P. Molnar wrote:

> I need to find a phrase in i text file in order to read a unique text
> string into a python3 application.
> 
> I have become stuck and Google has not been of any use.
> 
> Here is my attempt:

> import re
> 
> name = input("Enter Molecule Name: ")
> 
> name_in = name+'_apo-1acl.dlg'
> re.search("RMSD TABLE",name_in())

Stephen, you cannot throw random snippets of Python-lookalike code at a 
problem and expect it to work. When I wrote

> Work your way through a Python tutorial to pick up the basics

some two years ago I really meant it. If you can't be bothered have a look 
at preshrunk tools like grep, or find someone to write the code for you.
 
> Thanks in advance.

You're welcome.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cgi module help

[Peter Otten]

rmli...@riseup.net wrote:

> I have a question about the cgi module.
> 
> I'm trying to retrieve post data as a nested dictionary from client
> code.
> 
> For instance:
> 
> 
> 
> """client code"""
> from requests import sessions
> from datetime import datetime
> 
> session = sessions.Session()
> date = str(datetime.now())
> msg_id = 0001
> message = {"metadata": {"date": date, "id": msg_id}}
> session.post(data=message)

I made the above post to the server I found here

https://gist.github.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7

and got the following output:


$ python3 server.py 8080
INFO:root:Starting httpd...

INFO:root:POST request,
Path: /
Headers:
Host: localhost:8080
Content-Length: 25
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/2.2.1 CPython/2.7.6 Linux/3.13.0-170-generic



Body:
metadata=date=id

127.0.0.1 - - [13/Aug/2019 17:45:35] "POST / HTTP/1.1" 200 -

It looks like the data doesn't even get through.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class functions/staticmethod?

[Peter Otten]

James Hartley wrote:

> I am lacking in understanding of the @staticmethod property.
> Explanation(s)/links might be helpful.  I have not found the descriptions
> found in the Internet wild to be particularly instructive.  Given the code
> below:
> =8<--
> from collections import namedtuple
> 
> class Foo():
> Dimensions = namedtuple('Dimensions', ['height', 'width'])
> _dimensions = Dimensions(3, 4)
> 
> def dimensions():
> print('id = {}'.format(id(Foo._dimensions)))
> return Foo._dimensions

That works with the class as Foo.dimensions is just a function in Python 3, 
but not with an instance because Python will try to pass the instance as the 
first argument

>>> Foo.dimensions()
id = 140192821560880
Dimensions(height=3, width=4)
>>> Foo().dimensions()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: dimensions() takes 0 positional arguments but 1 was given

You can turn it into a static method

@staticmethod
def dimensions():
print('id = {}'.format(id(Foo._dimensions)))
return Foo._dimensions

>>> Foo.dimensions()
id = 139629779179056
Dimensions(height=3, width=4)
>>> Foo().dimensions()
id = 139629779179056
Dimensions(height=3, width=4)

or, when you are planning for subclases, into a classmethod:

$ cat staticmethod_demo.py
class Foo():
_dimensions = "foo-dimensions"

@classmethod
def class_dimensions(cls):
return cls._dimensions

@staticmethod
def static_dimensions():
return Foo._dimensions


class Bar(Foo):
_dimensions = "bar-dimensions"
$ python3 -i staticmethod_demo.py 
>>> Foo.class_dimensions(), Foo.static_dimensions()
('foo-dimensions', 'foo-dimensions')
>>> Bar.class_dimensions(), Bar.static_dimensions()
('bar-dimensions', 'foo-dimensions')

> 
> @staticmethod
> def dimensions1():
> print('id = {}'.format(id(_dimensions)))
> return _dimensions
> =8<--
> The class method Foo.dimensions() is capable of accessing class members,
> but Foo.dimensions1() cannot. What does the @staticmethod decorator really
> add?

You do not really need static methods; they work like module-level 
functions. They are more of a means to organize your code; by writing

class Foo:
   @staticmethod
   def bar(...): 
   do stuff

instead of

def foo_bar(...):
do stuff

class Foo:
pass

you make the mental association between the class and the function a bit 
stronger.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Object creation query

[Peter Otten]

mhysnm1...@gmail.com wrote:

>   All,
> 
>  
> 
> I think I am asking for the impossible here. But I will ask anyway.
> 
>  
> 
> I am using flask_sqlalchemy to build the tables and perform the queries,
> updates and insertions. I have multiple tables with the same structure
> with different names. A table called accounts which stores the name of the
> tables with the same structures. This is the important bits to know about.
> 
>  
> 
> I have a page called transactions. When I call this page, I can append
> different names to the end. For example:
> 
>  
> 
> Transactions/account1
> 
> Transactions/account2
> 
> Transactions/account3
> 
> .
> 
>  
> 
> In the view for transactions I am doing the following (code extract)
> 
>  
> 
> @app.route('/transactions/')
> 
> def transactions(account):
> 
> if accounts != "Transactions":
> 
> Accounts.query.filter_by(account_name =account).first_or_404()
> 
> tables = Accounts.query.all()
> 
> if account == 'Account1':
> 
> records = Account1
> 
> elif account == 'Account2':
> 
> records = Account2
> 
> records = records.query.order_by(records.date.desc)
> 
>  
> 
> as I am saving each model object into the same variable depending on the
> full URL name. I am wondering if there is a better way in doing this
> rather than using a list of if tests?

How about using only one table AccountEntries with an AccountId column
which would also be the primary key of the Accounts table. Then

# pseudo code

# look up account
wanted_accountId = select acountId 
   from Accounts 
   where name = account

# find records for account
records = select * from AccountEntries 
  where accountId = wanted_accountId
  order by date desc

That design would greatly simplify adding accounts 3 to, say, 30.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Create Logging module

[Peter Otten]

Sinardy Xing wrote:

> following is my main app
> 
> -- start here--
> from loggingme import logme
> 
> def say_hello(name, age):
> print('Hello {}, I am {}'.format(name, age))
> 
> #say_hello=logme(say_hello('Sinardy'))
> @logme
> say_hello('Tonny', 8)

Isn't this a SyntaxError? You can decorate functions, not function calls:

When Python finds a syntax error in your main script it won't proceed to run 
it and thus the bugs in modules that would be imported if the code were 
executed don't matter at this point.

Try

@logme
def say_hello(name, age):
print('Hello {}, I am {}'.format(name, age))

say_hello('Tonny', 8)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reading .csv data vs. reading an array

[Peter Otten]

Chip Wachob wrote:

> I tried it anyhow, with this being an example of my source data:
> 
> "Record Length",202,"Points",-0.005640001706,1.6363
> "Sample Interval",5e-09,s,-0.005639996706,1.65291
> "Trigger Point",1128000,"Samples",-0.005639991706,1.65291
> "Trigger Time",0.341197,s,-0.005639986706,1.60309
> ,,,-0.005639981706,1.60309
> "Horizontal Offset",-0.00564,s,-0.005639976706,1.6363
> ,,,-0.005639971706,1.65291
> ,,,-0.005639966706,1.65291
> ,,,-0.005639961706,1.6363
> .
> .
> .
> 
> Note that I want the items in the third and fourth column of the csv file
> for my time and voltage.
> 
> When I tried to use the unpack, they all came over as strings.  I can't
> seem to convert them selectively..

Try wrapping the reader like this:

 $ cat custom_reader.py
import csv
import io

data = """\
"Record Length",202,"Points",-0.005640001706,1.6363
"Sample Interval",5e-09,s,-0.005639996706,1.65291
"Trigger Point",1128000,"Samples",-0.005639991706,1.65291
"Trigger Time",0.341197,s,-0.005639986706,1.60309
,,,-0.005639981706,1.60309
"Horizontal Offset",-0.00564,s,-0.005639976706,1.6363
,,,-0.005639971706,1.65291
,,,-0.005639966706,1.65291
,,,-0.005639961706,1.6363
"""

def maybe_float(s):
try:
return float(s)
except ValueError:
return s

def myreader(*args, **kw):
reader = csv.reader(*args, **kw)
for row in reader:
yield [maybe_float(field) for field in row]

for row in myreader(io.StringIO(data)):
print(row)

$ python3 custom_reader.py 
['Record Length', 202.0, 'Points', -0.005640001706, 1.6363]
['Sample Interval', 5e-09, 's', -0.005639996706, 1.65291]
['Trigger Point', 1128000.0, 'Samples', -0.005639991706, 1.65291]
['Trigger Time', 0.341197, 's', -0.005639986706, 1.60309]
['', '', '', -0.005639981706, 1.60309]
['Horizontal Offset', -0.00564, 's', -0.005639976706, 1.6363]
['', '', '', -0.005639971706, 1.65291]
['', '', '', -0.005639966706, 1.65291]
['', '', '', -0.005639961706, 1.6363]

If you find that performance suffers more than you are willing to accept 
here's an alternative implementation of maybe_float() that may be faster for 
some inputs:

def maybe_float(s):
if s and s[:1] in " 0123456789+-":
try:
return float(s)
except ValueError:
return s
return s


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] replacing a loop

[Peter Otten]

johnf wrote:

> Hi folks,
> 
> 
> I have the following loop (actually repeated many times )

If you have repetetive code look into ways to parameterize it, like

def choices(rows, choices_column, keys_column):
...

> 
> def locChoices(self):
>  locDS = self.eslocation.getDataSet()
>  loc_Choices=['']
>  locKeys=[0]
>  for row in locDS:
>  loc_Choices.append(row['facility'])
>  locKeys.append(row['pkid'])
> 
> return loc_Choices,locKeys
> 
> where locDS is a tuple of dicts and a row is a dict.
> 
> Since I use a lot of similar loops to populate many dropdown controls I
> started investigating the use of list comprehensions.  But I can't
> figure out how to use them in this loop 

You need two loops in this case

choices = [""] + [row["facility"] for row in ds]
keys = [0] + [row["pkid"] for row in ds]

> and wonder if it will improve
> the performance. 

No, list comprehensions are unlikely to improve performance.

> The data is not very big - about a thousand rows -
> give or take.
> 
> So what do you guys think?

Are you sure (aka: did you measure that) building these lists takes a 
significant amount of time?
If there is a GUI involved any delay you notice is more likely to stem from 
filling the widgets than from preparing the lists. 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] os.is_file and os.is_dir missing from CPython 3.8.0b?

[Peter Otten]

Tom Hale wrote:

> I'm trying to use os.is_dir, but I'm not finding it or os.is_file.
> 
> What am I missing here?

Scroll up a bit in the documentation:

https://docs.python.org/3.8/library/os.html#os.DirEntry

Both is_file() and is_dir() are methods of the DirEntry object.

See also

https://docs.python.org/3.8/library/os.path.html#os.path.isfile

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Running Lib/test/test_shutil.py

[Peter Otten]

Tom Hale wrote:

> Hi all,
> 
> I hope this is the best place to ask (please let me know if there is a
> more appropriate list):
> 
> Checking out CPython v3.8.0b1, I'm trying to run:
> 
>% python Lib/test/test_shutil.py

Are you sure 

% python

invokes the 3.8 interpreter? 

> I'm getting:
> 
> Traceback (most recent call last):
>File "Lib/test/test_shutil.py", line 19, in 
>  from shutil import (make_archive,
> ImportError: cannot import name '_GiveupOnFastCopy' from 'shutil'
> (/usr/lib/python3.7/shutil.py)
> 

The traceback suggests that it may be 3.7.

Try

% ./python Lib/test/test_shutil.py

or, if you have installed python3.8

% python3.8 Lib/test/test_shutil.py

If you still get an error with a path into the /usr/lib/python3.7 stdlib
have a look at the PYTHONPATH environment variable.

> Am I trying to run the test file in the right way?
> 
> Context: I'm proposing to add /shutil.(sym)?link/ and want to start
> writing my tests with running the existing tests :)
> 
> Is there a doc that I've missed?
> 
> Cheers!
> 


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Error when trying to insert csv values into a sql table

[Peter Otten]

Cravan wrote:

> Here is the stack overflow link:
> https://stackoverflow.com/questions/56540292/error-when-trying-to-insert-csv-values-into-a-sql-table
> 
>  
> 
> I'm getting a weird error code when I try to store values from a csv into
> an sql table in a movie review assignment.

Like they say on stackoverflow: it very much looks like the movies table 
doesn't exist. Maybe you have forgotton a commit somewhere?

Please double-check that the table is actually created before you look for 
other less likely causes of your problem.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Collating date data from a csv file

[Peter Otten]

Cameron Simpson wrote:

> On 08May2019 21:04, Dave Hill  wrote:
>>I have a csv file which details the results of equipment tests, I
>>carry out PAT testing as a volunteer at a heriatge railway in N.
>>Wales. I want to extract how many items were tested on each test day.
>>So far I have generated a List of test dates, but I am now stalled at
>>how to efficiently count numbers tested on each date.
>>
>>Can I have a list of tuples, where one item is the date and the second
>>the count?
> 
> Not as such, because you can't modify a tuple (so you can't update the
> count part). But you could use a 2 element list.
> 
>>or is there a better construct?
> 
> Oh definitely. The easiest thing would be a defaultdict(int). Example:
> 
>   from collections import defaultdict
>   ...
>   by_date = defaultdict(int)
>   for row in csvdata:
> timestamp = row[1]  # based on your example data
> # get the date from the timestamp
> date = ...
> by_date[date] += 1
> 
> A defaultdict is a dict which magicly makes missing elements when they
> get access, using a factory function you supply. Here we're using "int"
> as that factory, as int() returns zero.

While this is easily adaptable if you want to keep more data...

by_date = defaultdict(list)  # rows grouped by date
for row in csvdata:
   date = ...
   by_date[date].append(row)

... for the simple case there is also collections.Counter:

def get_date(row):
return datetime.datetime.fromtimestamp(int(row[1])).date()

by_date = collections.Counter(map(get_date, csvdata))

# (date, freq) pairs ordered by frequency:
print(by_date.most_common())

> 
> I presume you've got the timestamp => date conversion sorted?
> 
> Cheers,
> Cameron Simpson 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help request ERROR installing beautifulsoup

[Peter Otten]

Dr. Luca T wrote:

> Hi,
> i'm new in python, i tried to install beautifulsoup but i had back this
> error:
> 
> ERROR: Complete output from command python setup.py egg_info:
> ERROR: Traceback (most recent call last):
>   File "", line 1, in 
>   File
>   "C:\Users\Luca\AppData\Local\Temp\pip-install-
u6zd808q\beautifulsoup\setup.py",
>   line 22
> print "Unit tests have failed!"
>   ^
> SyntaxError: Missing parentheses in call to 'print'. Did you mean
> print("Unit tests have failed!")?
> 
> ERROR: Command "python setup.py egg_info" failed with error code 1 in
> C:\Users\Luca\AppData\Local\Temp\pip-install-u6zd808q\beautifulsoup\
> 
> I use windows 10, python 3.7.3 and a minipc with 32-bit technology inside,
> can you help me telling me where i'm wrong please? 

Try installing bs4 instead of beautifulsoup to get a version that works with 
Python 3.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What does 'Bind Return Key' do?

[Peter Otten]

Matthew Polack wrote:

> Hi,
> 
> We're learning Python with PySimpleGUi and have used this example
> program...
> 
> 
https://github.com/PySimpleGUI/PySimpleGUI/blob/master/ProgrammingClassExamples/Win10%20versions/1d%20PSG%20(named%20input%20keys%20and%20catch%20errors).py
> 
> 
> There is a mystery command that says:
> 
> [sg.ReadButton('Submit', bind_return_key = False)]]
> 
> If I change this 'Bind_Return_Key' value from False to True...or in fact
> delete it entirely, it appears to make do difference whatsoever to the
> functioning of the program that I can see...
> 
> Could someone explain what the purpose of this could be? I'm guessing it
> has to have a reason to be there!

With

bind_return_key=True

hitting the  key has the same effect as clicking on the [Submit] 
button.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] help with colormode

[Peter Otten]

Mark Alderson wrote:

> hi
> 
> Ihave a very small program.  I want to cycle colours.  I cant set the
> colormode from 1 to 255
> 
> tried screen.colormode(255)
> 
> tells me screen is not defined.  the program works without the colormode,
> but i want to use it.
> 
> I just change the a and b variable values to generate new art.
> 
> -code-
> from turtle import Turtle
> t = Turtle()
> t.speed(0)
> 
> b = 180
> 
> a = 35
> 
> colormode(255)
> 
> t.color((55,55,55))
> for i in range(200):
> t.circle(i,a)
> t.right(b)
> t.circle(i,a)
> 
> 
> #input('Press any key to continue...')
> 
> -
> 
> ===error===
> Traceback (most recent call last):
>   File "H:\python\snowflake.py", line 9, in 
> screen.colormode(255)
> NameError: name 'screen' is not defined
> ===

The only name you import is Turtle, so you only have that (and the built-
ins). Fortunately you can get the screen from the Turtle, so:

from turtle import Turtle

ninja = Turtle()
ninja.speed(0)

screen = ninja.screen
screen.colormode(255)

b = 180
a = 35

for i in range(200):
ninja.color((i + 55, 55, 55))
ninja.circle(i, a)
ninja.right(b)
ninja.circle(i, a)

screen.exitonclick()


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What protocol to follow when need to pick either one from __getattr__ and __getattribute__ ?

[Peter Otten]

Arup Rakshit wrote:

> I read today 2 methods regarding the customizing the attribute
> access:__getattr__ and __getattribute__ from
> https://docs.python.org/3/reference/datamodel.html#special-method-names.
> What I understood about them is that __getattr__ is called when the
> requested attribute is not found, and an AttributeError is raised. But
> later is called everytime unconditionally. I wrote a simple 2 input
> calculator program, where only 2 operations are permitted Addition and
> Subtraction. Anything else will cause an not permitted error.
> 
> class OperationNotPermitted(AttributeError):
>  pass
> 
> class Calc:
>  def __init__(self, x, y):
>  self.x = x
>  self.y = y
> 
>  def __getattr__(self, name):
>  if name == "sum":
>  return self.x + self.y
>  elif name == 'minus':
>  return self.x - self.y
>  else:
>  raise OperationNotPermitted("operation {} is not
> permitted".format(name))
> 
> And here is a run down:
> 
> from customize_attr_access import *
> cal = Calc(12, 10)
> cal.sum
> 22
> cal.minus
> 2
> cal.mul
> Traceback (most recent call last):
>  Python Shell, prompt 5, line 1
>  # Used internally for debug sandbox under external interpreter
>  File "/Users/aruprakshit/python_playground/customize_attr_access.py",
> line 15, in __getattr__
>  raise OperationNotPermitted("operation {} is not
> permitted".format(name))
> customize_attr_access.OperationNotPermitted: operation mul is not
> permitted
> 
> If I replace __getattr__ with __getattribute__ I found the program works
> exactly same.

No, it doesn't, as __getattribute__ is called for x, and y, too.

 def __getattribute__(self, name):
 if name == "sum":
 return self.x + self.y
 elif name == 'minus':
 return self.x - self.y
 else:
 raise OperationNotPermitted("operation {} is not 
permitted".format(name))

Accessing cal.sum will therefore trigger a __getattribute__("x") call which in 
turn will raise an OperationNotPermitted("operation x ...") 
exception.


> Now my questions is in real world when  you have to pick
> between these 2 pair of special method which protocols a Python dev
> checks to pick either of the one? Is there any such thing, or either one
> is fine. Can anyone elaborate this to educate me please?

__getattribute__() is rarely needed, __getattr__() is useful when the
list of calculated attributes is open-ended and uniform (think proxy).
When there is a finite number of calculated attributes the best way to
implement them is usually a property:

class Calc:
def __init__(self, x, y):
self.x = x
self.y = y

@property
def sum(self):
return self.x + self.y

@property
def difference(self):
return self.x - self.y


> 
> doc said:
> 
>  > This method should either return the (computed) attribute value or
> raise an AttributeError exception.
> 
> Another question:
> 
> My question is that: Can I raise a domain error like
> OperationNotPermitted when raising instead of AttributeError ?

You just did ;) I don't think it's a good idea, though.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help

[Peter Otten]

fatima butt wrote:

> hi Peter,
> hope you are well.I am getting the following error when i am running the
> pygame shell script.I am using Acer SWIFT computer.my python version is
> 3.7.3 and pygame version is pygame 1.9.5
> 
> Traceback (most recent call last):
>   File "C:\Users\ammah\OneDrive\Documents\project1\myCode.py.py", line
>   166, in 
> draw_text(screen, str(score),18, WIDTH/2,10)
>   File "C:\Users\ammah\OneDrive\Documents\project1\myCode.py.py", line 28,
>   in draw_text
> font = pygame.font.Font(font_name, size)
> pygame.error: font not initialized

OK, you now have a different script, with a different error. Does that mean 
you resolved your previous problem?

Fine.

Regarding the new error I found the following hint

https://stackoverflow.com/questions/28517979/pygame-font-error

i. e. use 

pygame.init()

by entering the error message into a popular search engine ;)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: uploading images in pygame

[Peter Otten]

fatima butt wrote:

> the python version is 3.7.3
> computer is acer SWIFT
> The error I get is following:
> Traceback (most recent call last):
>   File "C:\Users\ammah\OneDrive\Documents\project1\myCode.py.py
> ", line 84, in 
> background =
> pygame.image.load(path.join(img_dir,"ship1.jpg")).convert()
> pygame.error: Couldn't open
> C:\Users\ammah\OneDrive\Documents\project1\ship1.jpg

Are you sure you have an image called 'ship1.jpg' in the
"C:\Users\ammah\OneDrive\Documents\project1" folder?

Double-check before you take other less likely problems into consideration.

If you can see the above file in your filemanager -- does the following 
script succeed?

with open(r"C:\Users\ammah\OneDrive\Documents\project1"), "rb"):
pass

If it doesn't, what does the traceback show?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Interoperating with Excel, was Re: Questions

[Peter Otten]

Diana Katz wrote:

> 1) Can you use python from excel? Or just export to excel?
> 2) I am trying to see if there's a way using python to automate all of
> this work that I need to do. I have to collect quarterly segment data for
> hundreds of public companies and go back at least 12-16 quarters. We use
> an aggregator like factset and they actually don't have this option
> available in an automated way. So I'm trying to see if there's a way to
> build this. Basically, I get my data from sec.gov and they have
> interactive data - they even have the data in excel (though it's a messy
> file and hard to read). I attached some of the steps and the data that i'd
> want to see. Basically i'd want the excel to look like:
> old to new quarters - going back 12 to 16 quarters (more if possible but
> not if it will stop the project).
>  Columns: 3/31/2017, 6/30/2017, 9/30/17, 12/31/17, 3/313/2018...
> Rows:
> Sales for segment A
> Sales for Segment b
> Sales for SEgment C
> …(for as many segments as they have)
> 
> Earnings for Segment A
> .Earnings for Segment B
> 
> Depreciation for Segment A
> Depreciation for Segment B
> Depreciation for Segment C...

These look like "pivot tables" which are well supported by Excel.
I expect that this is easy to automate with a little bit of Basic.

Of course you can build these tables with a Python script if you feel more 
comfortable in Python. Then either write them into csv files ("comma 
separated value", supported by the standard library) 

https://docs.python.org/3/library/csv.html

which can be read by Excel -- or use a dedicated library. Google came up 
with

https://xlsxwriter.readthedocs.io/

> I included where I get the data in the attached document.

Attachments don't make it to the list. You have to provide a link or (if 
they're small and text-only) paste your data and scripts into the message 
body.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Encrypting shipped sqlite db in app directory

[Peter Otten]

Ali M wrote:

> I want to encrypt my sqlite databases which are shipped with the app in
> it's directory, so that the user can't modify or use it elsewhere, and
> they will only be accessible for the app to read from, how can i do that?

Assuming you had an encrypted database, where would you put the password?
You cannot give it to the user because then he can decrypt the database, and 
if you put it into the app you are effectively giving it to the user.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Peter Otten]

Matthew Herzog wrote:

> I have a Python3 script that reads the first eight characters of every
> filename in a directory in order to determine whether the file was created
> before or after 180 days ago based on each file's name. The file names all
> begin with MMDD or erased_MMDD_etc.xls. I can collect all these
> filenames already.
> I need to tell my script to ignore any filename that does not conform to
> the standard eight leading numerical characters, example: 20180922 or
> erased_20171207_1oIkZf.so.
> Here is my code.
> 
> if name.startswith('scrubbed_'):
> fileDate = datetime.strptime(name[9:17], DATEFMT).date()
> else:
> fileDate = datetime.strptime(name[0:8], DATEFMT).date()
> 
> I need logic to prevent the script from 'choking' on files that don't fit
> these patterns. The script needs to carry on with its work and forget
> about non-conformant filenames. Do I need to add code that causes an
> exception or just add an elif block?

Personally I would use a try...except clause because with that you can 
handle invalid dates like _etc.xls gracefully. So

ERASED = "erased_"


def strip_prefix(name):
if name.startswith(ERASED):
name = name[len(ERASED):]
return name


def extract_date(name):
datestr = strip_prefix(name)[:8]
return datetime.datetime.strptime(datestr, DATEFMT).date()


for name in ...:
try:
file_date = extract_date(name)
except ValueError:
pass
else:
print(file_date)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] removing xml elements with ElementTree

[Peter Otten]

street.swee...@mailworks.org wrote:

> An opportunity to work in Python, and the necessity of working with some
> XML too large to visualize, got me thinking about an answer Alan Gauld had
> written to me a few years ago
> (https://mail.python.org/pipermail/tutor/2015-June/105810.html).  I have
> applied that information in this script, but I have another question :)
> 
> Let's say I have an xml file like this:
> 
> -- order.xml 
> 
> 
> Bob
> 321 Main St
> 
> 
> D20
> 4
> 
> 
> CS211
> 1
> 
> 
> BL5
> 7
> 
> 
> AC400
> 1
> 
> 
> 
> 
> -- end order.xml 
> 
> Items CS211 and AC400 are not valid items, and I want to remove their
>  nodes.  I came up with the following (python 3.6.7 on linux):
> 
>  xml_delete_test.py 
> 
> import os
> import xml.etree.ElementTree as ET
> 
> hd = os.path.expanduser('~')
> inputxml = os.path.join(hd,'order.xml')
> outputxml = os.path.join(hd,'fixed_order.xml')
> 
> valid_items = ['D20','BL5']
> 
> tree = ET.parse(inputxml)
> root = tree.getroot()
> saleslines = root.find('saleslines').findall('salesline')
> for e in saleslines[:]:
> if e.find('item').text not in valid_items:
> saleslines.remove(e)
> 
> tree.write(outputxml)
> 
> -- end xml_delete_test.py --
> 
> The above code runs without error, but simply writes the original file to
> disk.  The desired output would be:
> 
> -- fixed_order.xml 
> 
> 
> Bob
> 321 Main St
> 
> 
> D20
> 4
> 
> 
> BL5
> 7
> 
> 
> 
> 
> -- end fixed_order.xml 
> 
> What I find particularly confusing about the problem is that after running
> xml_delete_test.py in the Idle editor, if I go over to the shell and type
> saleslines, I can see that it's now a list of two elements.  I run the
> following:
> 
> for i in saleslines:
> print(i.find('item').text)
> 
> and I see that it's D20 and BL5, my two valid items.  Yet when I write
> tree out to the disk, it has the original four.  Do I need to refresh tree
> somehow?
> 
> Thanks!

First of all, thank you for this clear and complete problem description!

> saleslines = root.find('saleslines').findall('salesline')

Here findall()

returns a new list of matches which is completely independent of the element 
tree. Therefore

> saleslines.remove(e)

will remove the element e from this indepent list, and only from that.
To remove an element from the tree you have to know its parent, and then

parent_element.remove(child_element)

will actually modify the tree. 

In your case the parent is always , so you can restrict yourself 
to its children:

saleslines = root.find('saleslines')
for e in saleslines.findall('salesline'):
if e.find('item').text not in valid_items:
saleslines.remove(e)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Merge a dictionary into a string

[Peter Otten]

Valerio Pachera wrote:

> Consider this:
> 
> import collections
> d = OrderedDict(a='hallo', b='world')
> 
> I wish to get a single string like this:
> 
> 'a "hallo" b "world"'
> 
> Notice I wish the double quote to be part of the string.
> In other words I want to wrap the value of a and b.
> 
> I was thinking to use such function I created:
> 
> def mywrap(text, char='"'):
> return(char + text + char)
> 
> I can't think anything better than
> 
> s = ''
> for k, v in d.items():
>  s += ' '.join( (k, mywrap(v)) ) + ' '
> 
> or
> 
> s = ''
> for k, v in d.items():
> s += k + ' ' + mywrap(v) + ' '
> 
> What do you think?
> It's fine enough but I wonder if there's a better solution.

In Python 3.6 and above you can use f-strings:

>>> d = dict(a="hello", b="world")
>>> " ".join(f'{k} "{v}"' for k, v in d.items())
'a "hello" b "world"'

By the way, are you sure that the dictionary contains only strings without 
spaces and '"'?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Peter Otten]

Glenn Dickerson wrote:

> class Student():
> 
> def__init__(self, name, major, gpa, is_on_probation):
> self.name = name
> self.major = major
> self.gpa = gpa
> self.is_on_probation = is_on_probation
> 
> 
> import Student
> student1 = Student('Jim', 'Business', 3.1, False)
> student2 = Student('Pam', 'Art', 2.5, True)
> print(student1.name)
> print(student2.gpa)
> 
> I entered this in IDLE and it failed. Any advice?

First of all, try to be as precise as you can with your error descriptions.
What was the exact error, what looked the traceback like?

Do not retype, use cut-and-paste to put them into your post.

Was it something like

  File "Student.py", line 3
def__init__(self, name, major, gpa, is_on_probation):
^
SyntaxError: invalid syntax

Then the problem is the missing space between the keyword "def" and the 
method name "__init__".

Or was it

Traceback (most recent call last):
  File "student.py", line 10, in 
import Student
ImportError: No module named 'Student'

That's because Student is a class rather than a module, and you neither
need to nor can import it directly. If you remove the import statement your 
code should work.

But wait, there's another option. If you saw

Traceback (most recent call last):
  File "Student.py", line 10, in 
import Student
  File "Student.py", line 11, in 
student1 = Student('Jim', 'Business', 3.1, False)
TypeError: 'module' object is not callable

then a "Student" module was imported successfully. However, as it has the 
same name as your class, the name "Student" is now bound to the module, and 
unlike the class a module cannot be called. The solution then is to rename 
the module, from Student.py to student.py, say, as lowercase module names 
are the preferred convention anyway.

So there are at least three possible problems in your tiny and almost 
correct code snippet. 

In a script that does some actual work the number of possible problems 
explodes, and that's why most of us don't even start to debug a piece of 
code without a detailed error description and a traceback -- it's usually a 
waste of time.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] My problem in simple terms

[Peter Otten]

Edward Kanja wrote:

> Hi there ,
> Earlier i had sent an email on how to use re.sub function to eliminate
> square brackets. I have simplified the statements. Attached txt file named
> unon.Txt has the data im extracting from. The file named code.txt has the
> codes I'm using to extract the data.The regular expression works fine but
> my output has too many square brackets. How do i do away with them thanks.

The square brackets appear because re.findall() returns a list. If you know 
that there is only one match or if you are only interested in the first 
match you can extract it with

first = re.findall(...)[1]

This will of course fail if there is no match at all, so you have to check 
the length first. You can also use the length check to skip the lines with 
no match at all, i. e. the line appearing as

[] [] []

in your script's output.

Now looking at your data -- at least from the sample it seems to be rather 
uniform. There are records separated by "---..." and fields separated by 
"|". I'd forego regular expressions for that:

$ cat code.py
from itertools import groupby

def is_record_sep(line):
return not line.rstrip().strip("-")

with open("unon.txt") as instream:
for sep, group in groupby(instream, key=is_record_sep):
if not sep:
record = [
[field.strip() for field in line.split("|")]
for line in group if line.strip().strip("|-")
]
# select field by their position in the record
names = record[0][1]
station = record[0][2]
index = record[1][1]
print(index, names, station, sep=", ")
$ python3 code.py 
3648, Rawzeea NLKPP, VE11-Nairobi
1007, Pattly MUNIIZ, TX00-Nairobi


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Variable change within library depending on user input.

[Peter Otten]

Harry Oneill wrote:

> Hey there everybody hope your doing great.
> 
> I was here a few months ago and got pointed in the right direction very
> kindly by one of the tutors. Im a little stuck again now and have been
> researching for a while and can't come up with a solution to my problem.
> 
> The below program is designed to take two user inputs, item purchase
> location and sale location and then to compare some dictionaries of those
> locations and calculate the most profitable item to buy.
> 
> Initially i designed this to work with just two locations and am now
> struggling to add a third.

Do you still have the version that works with two locations?
Then start from that with the following steps:

(1) Extract the code that uses a buying and a selling, and finds the most 
profitable good into a function

def find_most_profitable(buying_dict, selling_dict):
... # code adapted from the version that works with two locations

(2) Write the main script that picks a buying dict and a selling dict using 
user input. The simplest approach is to put the buying dicts into another 
dict:

buying_dicts = {"olisar": buying_olisar, "levski": buying_levski, "156": 
buying_arc_corp_mining_area_157}
selling_dicts = ...

def pick_dict(choices, prompt):
print(prompt)
while True:
print("Pick one of", ", ".join(sorted(choices)))
choice = input("> ")
if choice in choices:
return choices[choice]

buying =  pick_dict(
buying_dicts,
"Were are you planning to buy your goods?"
)
selling = pick_dict(
selling_dicts,
"Where are you planning to sell your goods?"
)

find_most_profitable(buying, selling)

As a bonus the pick_dict() function checks the validity of your choice and 
lets you repeat your input if you made a spelling error.

> However they produce this error that i am unsure how to resolve.
> 
> Traceback (most recent call last):
>   File "/home/floppypoppy/PycharmProjects/Star Citizen Trading
>   Program/Star Citizen trading NEW.py", line 33, in 
> tradable_items = common(user_purchase_location_1,
> user_sale_destination_1)
>   File "/home/floppypoppy/PycharmProjects/Star Citizen Trading
>   Program/Star Citizen trading NEW.py", line 32, in common
> return set(curr.keys()).intersection(set(other.keys()))
> AttributeError: 'str' object has no attribute 'keys'

Now why did you see the traceback?

>  Sorting the dict into ascending value instead of alphabetical
> sort_buying_olisar = sorted(buying_olisar, key=lambda tup: tup[1])


Sorting a dict implies iteration, and iterating over a dict produces the 
keys:

>>> d = {"az": 2, "by": 1, "cx": 3}
>>> list(d)
['cx', 'by', 'az']

Therefore

>>> sorted(d, key=lambda k: k[1])
['cx', 'by', 'az']

sorts the keys by the second character.
To sort the (key, value) pairs by value you need to be explicit:

>>> sorted(d.items(), key=lambda kv: kv[1])
[('by', 1), ('az', 2), ('cx', 3)]

If you are using Python 3.7 (and inofficially 3.6 I think) you can convert 
this back into a dict with

items = sorted(d.items(), key=lambda kv: kv[1])
ordered = dict(items)

In older Pythons you may need

ordered = collections.OrderedDict(items)

instead.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove soft line break

[Peter Otten]

Valerio Pachera wrote:

[Me:]

>> def merge_lines(lines):
>> lines = (line.rstrip("\n") for line in lines)
>> accu = [next(lines)]
>> for line in lines:
>> if line.startswith(" "):
>> accu.append(line[1:])
>> else:
>> yield "".join(accu) + "\n"
>> accu = [line]
>> yield "".join(accu) + "\n"

> I think it could be solved in a much easier way.

> content=f.read().replace('\n ', '')

That is indeed much simpler than my suggestion.

> What am I missing?

Nothing. My attempt to keep memory usage low led to a solution which is more 
complex than your slurp-it-in, fix it, spit-it-out.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Only appending one object to list, when I am expecting more than 1

[Peter Otten]

AdamC wrote:

> I'm creating lots of objects from json in a file. Part of reading the json
> back means that it iterates over the file and loads a json object and then
> creates the object from the dictionary.
> 
> This is my  file:
> 
> {"name": "Dwarf Fortress", "platform": "steam", "dateAdded":
> "2019:02:25:16:56:1551113768", "tpe": "2019:02:21:13:49:1550756942"}
> {"name": "Jaws", "platform": "Netflix", "dateAdded":
> "2019:02:25:16:56:1551113768", "tpe": "2019:02:21:13:49:1550756960"}
> {"name": "Wargames", "platform": "CLI", "dateAdded":
> "2019:02:25:16:59:1551113984", "tpe": "Game"}
> 
> and these are the functions that help load that file:
> 
> media = []
> 
> def loadFile():
> filename = input('Filename? ')
> f = open(filename, 'r')
> createObjects(f)
> 
> def createObjects(f):
> '''Takes a file object and iterates through entries, passing them to
> create
> object, depending on what object it is.'''
> for line in f:
> count = count + 1
> data = json.loads(line)
> print(type(data['tpe']))
> name = data['name']
> platform = data['platform']
> dateAdded = data['dateAdded']
> tpe = data['tpe']
> if data['tpe'] == 'Game':
> a = createGame(name, platform, dateAdded,tpe)
> game = {a: tpe}
> media.append(game)
> if data['tpe'] == 'Film':
> a = createFilm(name, platform, dateAdded,tpe)
> film = {a: tpe}
> media.append(film)
> # For some reason I'm only getting one object at a time now when
> appending to media
> print(len(media))
> 
> def createGame(name, platform, dateAdded, tpe):
> return Game(name, platform, dateAdded)
> 
> def createFilm(name, platform, dateAdded, tpe):
> return Film(name, platform, dateAdded)
> 
> (This isn't the order that the functions are held in the module).
> Why would I only get one object in media, even though all three are
> created?

Only one record in your jsonl sample has a 'tpe' handled by createObjects(). 
To make it easier to catch this problem (invalid or unsuspected data) I 
suggest that you rewrite the

> if data['tpe'] == 'Game':
> a = createGame(name, platform, dateAdded,tpe)
> game = {a: tpe}
> media.append(game)
> if data['tpe'] == 'Film':
> a = createFilm(name, platform, dateAdded,tpe)
> film = {a: tpe}
> media.append(film)

part of your code as

if data['tpe'] == 'Game':
a = createGame(name, platform, dateAdded,tpe)
game = {a: tpe}
media.append(game)
elif data['tpe'] == 'Film':
a = createFilm(name, platform, dateAdded,tpe)
film = {a: tpe}
media.append(film)
else:
# Instead of the error message you may also raise an exception.
print(
"Warning: unknown tpe={!r}".format(data["tpe"]),
file=sys.stderr
)

The unconditional else allows you to ensure that every line will be handled.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] import failure

[Peter Otten]

Alex Kleider wrote:

> (p2) alex@one:$ ./tests/run_data_tests.py

Here you let the script decide which interpreter to pick...

> Traceback (most recent call last):

> ImportError: No module named tlslite.utils

...and it looks like it's not the one you'd like to have:

> (p2) alex@one:$ python
> Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
> [GCC 7.3.0] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 import tlslite
 exit()

> Any suggestions as to what the problem might be or how to investigate
> further would be very much appreciated.

Try

(p2) alex@one:$ python ./tests/run_data_tests.py



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] text processing lines variable content

[Peter Otten]

ingo janssen wrote:

> 
> 
> On 07/02/2019 11:08, Peter Otten wrote:
>> replace the sequence of tests with dictionary lookups
> 
> updated the gist a few times, now I could pre calculate the slices to be
> taken per line, but will there be much gain compared to the copping from
> the left side of the list?

Sorry, I don't understand the question.


Looking at your code

> if action == "%i":
> lbl = function[action](content[action])

you really do not need the function[action] lookup here because you know 
that the result will always be f_number. Likewise you could bind 
content["%i"] to a name, labels, say, and then write

if action == "%s":
lbl = f_number(labels)

which I find much more readable. 

A lookup table only makes sense if provides all necessary information. I 
tried to apply the idea to one of your gist versions:

def set_lbl(items):
global lbl
lbl = f_number(items)

def set_w(items):
global v
v = f_number(items)

def set_f(items):
global f
f = f_number(items)

def set_mx(items):
global mx
mx = mx_value_array(items, f)

function = {
"%i" : set_lbl,
"%w" : set_w,
"%s" : set_f,
"%a" : set_mx,
"%q" : f_vector,
"%r" : f_value,
"%p" : lambda items: f_vector_array(items, v),
"%P" : lambda items: f_vector_array(items, v),
"%o" : lambda items: f_value_array(items, v),
"%m" : f_value,
"%g" : f_number,
"%E" : f_value,
"%e" : lambda items: f_value_array(items, f),
"%F" : f_value,
"%A" : lambda items: f_value_array(items, mx + 1),
"%f" : lambda items: f_value_array(items, f),
"%t" : lambda items: f_nested_value_array(items, f),
"%l" : lambda items: f_vector_array(items, f),
"%n" : lambda items: f_value_array(items, f),
"%v" : f_value,
"%c" : f_vector,
"%C" : f_vector
} 

order = "%i %q %r %w %p %P %o %m %g %E %s %e %F %a %A %f %t %l %n %v %c %C"
order = re.findall("%[a-z]",order,re.M|re.I)
content = {}

actions = []

for i in order:
items = content[i] = []
actions.append(partial(function[i], items))

for points, line in enumerate(open("vorodat.txt.vol",'r'), 1):
line = line.strip()
line = line.split(" ")
for action in actions:
action()

However, while the loop is rather clean now the rest of the code is 
sprinkled with implicit arguments and thus much worse than what you have.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] text processing lines variable content

[Peter Otten]

ingo janssen wrote:

> 
> On 07/02/2019 09:29, Peter Otten wrote:
>> Where will you get the order from?
> 
> Peter,
> 
> the order comes from the command line. 

Then my one-function-per-format approach won't work.

> I intend to call the python
> program with the same command line options as the Voro++ program. Make
> the python program call the Voro++ and process its output.
> 
> one command line option contains a string for formatting the output.
> That is what I use for order.
> 
> #all output formatting options
> order = "%i %q %r %w %p %P %o %m %g %E %s %e %F %a %A %f %t %l %n %v %c
> %C" order = re.findall("%[a-z]",order,re.M|re.I)
> for i, line in enumerate(open("vorodat.vol",'r')):
>points = i
>line = line.strip()
>line = line.split(" ")
>for action in order:
>  if action == "%i":
>try:
>  lbl = f_label(label)
>except NameError as e:
>   lbl = f_number(label)
>   label=[lbl]

Personally I would avoid the NameError and start with empty lists. If you 
manage to wrap all branches into functions with the same signature you can 
replace the sequence of tests with dictionary lookups. Here's a sketch:


# the f_...() functions take a parsed line and return a value and the
# as yet unused rest of the parsed line

labels = []
points = []
...
def add_label(parts):
   label, rest = f_label(parts)
   labels.append(label)
   return rest

def add_point(parts):
point, rest = f_vector(parts)
points.append(point)
return rest

def add_point(parts):
global width
width, rest = f_width(parts)
return rest

lookup_actions = {
"%i": add_label,
"%q": add_point,
"%w": set_width,
...
}

actions = [lookup_actions[action] for action in order]

with open("vorodat.vol") as instream:
for points, line in enumerate(instream, 1):  # as per Mark's advice
width = None  # dummy value to provoke error when width
  # is not explicitly set
parts = line.split()
for action in actions:
parts = actions(parts)


>  elif action == "%q":
>try:
>  f_vector(point)
>except NameError as e:
>point = [f_vector(point)]
>  elif action == "%r":
>try:
>  f_value(radius)
>except NameError as e:
>  radius=[f_value(radius)]
> etc.
> 
> order is important as %w tells me how long %p, %P and %o will be. This
> varies per line.
> 
> I'll look into what you wrote,

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] text processing lines variable content

[Peter Otten]

ingo janssen wrote:

> depending on how the input file is created data packet a can be in an
> other position for every line.
> figured out how to do it though
> 
> order=[a,b,e,d...]
> for i in lines:
>i=i.split(" ")
>  for j in order:
>if j = a:
> use function for processing data chunk a
>elseif j = b:
>  use proper function for processing data type b
>...

Where will you get the order from? If you plan to specify it manually, e. g.

lookup_steps = {
"foo": [a, b, c, ...],
"bar": [a, a, f, ...],
}
fileformat =  sys.argv[1]
steps = lookup_steps[fileformat]
...
for line in lines:
for step in steps:
if step == a:
...
elif step == b:
...

then I recommend storing one function per file format instead:

def process_foo(line):
...  # process one line in foo format

def process_bar(line):
...

lineprocessors = {
"foo": process_foo,
"bar": process_bar,
}
fileformat =  sys.argv[1]
process = lineprocessors[fileformat]
...
for line in lines:
process(line)

That way you deal with Python functions instead of a self-invented 
minilanguage.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] help

[Peter Otten]

Sonia Miglani wrote:

> Hi Team,
> 
> I am learning puthon and trying the following code.
> 
> But getting the following error.
> 
> Please help me in knowing the code in better way.
> 
> 
> OS Linux
> Python version 2.7.13
> 
> 
> 
> def  demo(s, exclaim):
> #"""
>  #   Returns the string 's' repeated 3 times.
>   #  If exclaim is true, add exclamation marks.
> 
> 
> result = s + s + s
> if exclaim:
> result = result + '!!!'
> return result
> 
> 
> def main():
> print demo('Yay', False)  ## YayYayYay
> print demo('Woo Hoo', True)   ## Woo HooWoo HooWoo Hoo!!!
> 
> 
> Error:
> ./python2.py: line 1: syntax error near unexpected token `('

That is not a Python error, that's a complaint of your shell.
If you make a Python script executable you also have to insert the proper 
hash-bang line. In the case of Python 2

#!/usr/bin/python2

will probably work. Example shell session:

$ cat tmp.py
def  demo():
print "heureka"

demo()
$ ./tmp.py
./tmp.py: line 1: syntax error near unexpected token `('
./tmp.py: line 1: `def  demo():'
$ cat tmp2.py
#!/usr/bin/python2

def  demo():
print "heureka"

demo()
$ ./tmp2.py
heureka
$ 


> ./python2.py: line 1: `def  demo(s,exclaim):
> 
> 
> 
> 
> 
> Regards
> Sonia
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove soft line break

[Peter Otten]

Valerio Pachera wrote:

> 
> I have a file with row that split at the 80th character.
> The next row start with a blank space, meaning that i part of the previous
> row.
> 
> Example:
> 
> Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam non justo
> enim. Viv
>  amus dapibus quis neque vitae ornare. Pellentesque at pharetra sapien, id
>  eleif end lacus. Nullam ut semper enim, vulputate venenatis justo.
>  Vestibulum vehicul a dolor sit amet ultricies vulputate. Aenean lobortis,
>  nulla eu scelerisque hen
> 
> What do you suggest to get the text on a single line?

Neglecting the corner cases:

$ cat mergelines.py 
def merge_lines(lines):
lines = (line.rstrip("\n") for line in lines)
accu = [next(lines)]
for line in lines:
if line.startswith(" "):
accu.append(line[1:])
else:
yield "".join(accu) + "\n"
accu = [line]
yield "".join(accu) + "\n"


SAMPLE =  """\
foo bar baz
  ham spam
alpha beta
  gamma
 delta
  epsilon
""".splitlines(True)

for line in merge_lines(SAMPLE):
print(line, end="")
$ python3 mergelines.py 
foo bar baz ham spam
alpha beta gammadelta epsilon

I hope this isn't homework ;)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python-files

[Peter Otten]

Asad wrote:

> Hi All ,
> 
>   I tried the following code  :
> 
> parser = argparse.ArgumentParser()
> parser.add_argument("first")
> parser.add_argument("second", nargs="?")
> args = parser.parse_args()
> print("first:", args.first)
> 
> print("second:", args.second)
> 
> When I execute the script it gives error :
> 
> python python_json_20001_oratest_v1.py "file1"
> ('first:', 'file1')
> ('second:', None)
> Traceback (most recent call last):
>   File "test.py", line 211, in 
> with open(args.second, 'r') as f :
> TypeError: coercing to Unicode: need string or buffer, NoneType found
> 
> 
> if I see in line number 211 it with open(args.second, 'r') as f :
> 
> try :
> with open(args.second, 'r') as f :
>  for line in f:
> print line
> except IOError:
>  print "The default error is err-1 because file2 was not provided
>  "

How do you know that the file was not provided? The name might have been 
misspelt or the user lacks the permission to read it.

> Does that mean my try and except block is not working because if
> args.second  is None as in this case then it should print "The default
> error is err-1 because file2 was not provided "
> 
> Please advice ,

If you just want to terminate the script with a helpful message you should 
make the second argument mandatory, too:

parser = argparse.ArgumentParser()
parser.add_argument("first")
parser.add_argument("second")
args = parser.parse_args()

That way the parse_args() call will terminate the script and the user will 
see an error message immediately.

If for some reason you want to keep the argument optional you can check for 
None before trying to open the file:

if args.second is not None:
try:
with open(args.second, 'r') as f :
for line in f:
print line
except IOError as err:
print err  # print actually what happened, not what you guess
else:
print "File 'second' was not provided on the command line"



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python - files

[Peter Otten]

Cameron Simpson wrote:

> Mats has mentioned the modules getopt and argparse etc. These are
> primarily aimed at option parsing ("-v", "-o foo"). Your situation
> occurs _after_ the option parsing (in your case, there are no options).

Not argparse. The main advantage over optparse is its handling of positional 
arguments. Your custom logic 

>   def main(argv):
> cmd = argv.pop(0)   # collect the command word
> badopts = False
> # mandatory first argument
> if not argv:
>   print("%s: missing first argument" % cmd, file=sys.stderr)
>   badopts = True
> else:
>   first = argv.pop(0)
>   # optional second argument
>   if argv:
> second = argv.pop(0)# explicit argument 2, use it
>   else:
> second = None   # or some otherdefault
>   if argv:
> print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr)
> badopts = true
> if badopts:
>   print("%s: invalid invocation, aborting" % cmd, file=sys.stderr)
>   return 2
> ... work with first and second ...

can roughly be replicated with the two lines

parser.add_argument("first")
parser.add_argument("second", nargs="?")

A working script:

$ cat test.py
#!/usr/bin/python3
import argparse

def main():
parser = argparse.ArgumentParser()

parser.add_argument("first")
parser.add_argument("second", nargs="?")

args = parser.parse_args()

print("first:", args.first)
print("second:", args.second)

if __name__ == "__main__":
main()

$ ./test.py
usage: test.py [-h] first [second]
test.py: error: the following arguments are required: first

$ ./test.py -h
usage: test.py [-h] first [second]

positional arguments:
  first
  second

optional arguments:
  -h, --help  show this help message and exit

$ ./test.py ONE
first: ONE
second: None

$ ./test.py ONE TWO
first: ONE
second: TWO

$ ./test.py ONE TWO THREE
usage: test.py [-h] first [second]
test.py: error: unrecognized arguments: THREE

Argparse makes a usable standard command line interface easy to set up (if 
you need non-standard behaviour it gets a bit harder).

There is also a companion module argcomplete (not in the stdlib) that 
enables autocompletion.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] exporting lists into CSV issue.

[Peter Otten]

mhysnm1...@gmail.com wrote:

> Now I am
> trying to export to a CSV file. There is no syntax or logical errors I can
> see. The information is being exported to the CSV. But when I bring it
> into Excel. I am getting a blank row between each row.

> with open ('books-list.csv', 'w') as wf:

Try switching off newline conversion:

with open('books-list.csv', 'w', newline="") as wf:
   ...

The csv.writer() delimits records with "\r\n", and on Windows open() 
converts "\n" to "\r\n" by default. Therefore you probably (no Window handy 
to to confirm) end up with "\r\r\n" between the rows.

See also .

> writer = csv.writer(wf)
> writer.writerows(books)

At this point the file is already closed by the context manager, no need for 
an extra

> wf.close()


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Importing XML files.

[Peter Otten]

mhysnm1...@gmail.com wrote:

> I am trying to import ITunes XML files. I have been doing some reading and
> I am somewhat confused with some XML terminology.

> What is:
> 
>  
> 
> Tag – I think it is the  in the itunes library file.
> 
> Text = The text between the 
> 
> Attrib  -- don’t know.
> 
>  
> 
> If I am correct in my thinking. If you look up all the tags using the xml
> module. How do you then isolate all the name text? I do not have any
> working code at this present time. I have been playing with the xml
> methods and reading some example tutorials. But I am stumped with the
> above terminology which is a small road block. Below is an example extract
> of my xML ITunes to see if this helps. I am doing this in Python 3.7 for
> Windows 10.
> 
>  
> 
> 
> 
>  "http://www.apple.com/DTDs/PropertyList-1.0.dtd;>
> 
> 

You may be lucky in that you can avoid the hard way outlined by Alan -- 
Python's stdandard library includes a module that handles Apple's plist 
format. I tried your sample, and the module seems to turn it into nested 
dicts:

>>> import plistlib, pprint
>>> with open("sample.xml", "rb") as f:
... data = plistlib.load(f)
... 
>>> pprint.pprint(data, width=70)
{'Application Version': '12.8.0.150',
 'Date': datetime.datetime(2019, 1, 14, 3, 56, 30),
 'Features': 5,
 'Library Persistent ID': 'F2D33B339F0788F0',
 'Major Version': 1,
 'Minor Version': 1,
 'Music Folder': 'file:///Volumes/Itunes/iTunes/iTunes%20Media/',
 'Show Content Ratings': True,
 'Tracks': {'6493': {'Album': 'In Her Sights (Unabridged)',
 'Album Artist': 'Robin Perini',
 'Artist': 'Robin Perini',
 'Artwork Count': 1,
 'Bit Rate': 64,
 'Comments': "Jasmine 'Jazz' Parker, "
 'Jefferson County SWAT’s only '
 'female sniper, can thread the '
 'eye of a needle with a bullet. '
 'But she carries with her a '
 'secret from her past',
[snip]
>>> data["Date"].isoformat()
'2019-01-14T03:56:30'
>>> list(data["Tracks"].values())[0]["Artist"]
'Robin Perini'

Looks good, except for the mojibake -- but that was already in your email.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Implementation of list comparison operators

[Peter Otten]

David Rock wrote:

> 
>> On Jan 17, 2019, at 13:40, Peter Otten <__pete...@web.de> wrote:
>> 
>> David Rock wrote:
>> 
>>> 
>>> Isn’t this a bit artificial, though?  The reason this is False is
>>> because
>>> you explicitly tell it to return False when using equality.  That’s not
>>> the same thing as using __eq__ without overriding it’s behavior
>>> internally.
>> 
>> Sorry, I don't understand that argument. My point wasn't whether it's a
>> good idea to write objects that compare unequal to themselves -- such
>> objects already exist:
>> 
>>>>> nan = float("nan")
>>>>> nan == nan
>> False
>> 
>> I only warned that a list containing such an object does not meet the
>> intuitive expectation that list_a == list_b implies that all items in
>> list_a compare equal to the respective items in list_b.
> 
> It’s certainly a valid warning.  My confusion came from you using an
> arbitrary example of creating a class that breaks the logic in an override
> rather than one that already exists as a concrete example.  To me, your
> class example looked contrived.
> 
> What is it about the float(“nan”) example that makes this break?
> 
> In [5]: nan = float("nan”)
> 
> In [6]: type(nan)
> Out[6]: float
> 
> In [7]: nan == nan
> Out[7]: False
> 
> In [8]: a = 1.1
> 
> In [9]: a ==a
> Out[9]: True
> 
> In [10]: type(a)
> Out[10]: float
> 
> both a and nan are floats, so why does a == a work, but nan == nan
> doesn’t?

It does "work", it's only produces a result you didn't expect ;) 
Python just follows the standard here

https://en.wikipedia.org/wiki/IEEE_754
https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN

Also:

https://stackoverflow.com/questions/10034149/why-is-nan-not-equal-to-nan


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Implementation of list comparison operators

[Peter Otten]

David Rock wrote:

> 
>> On Jan 17, 2019, at 12:39, Peter Otten <__pete...@web.de> wrote:
>> 
>> One obscure detail of the implementation of list equality:
>> 
>> In Python an object can be unequal to itself:
>> 
>>>>> class A:
>> ... def __eq__(self, other): return False
>> ...
>>>>> a = A()
>>>>> a == a
>> False
> 
> Isn’t this a bit artificial, though?  The reason this is False is because
> you explicitly tell it to return False when using equality.  That’s not
> the same thing as using __eq__ without overriding it’s behavior
> internally.

Sorry, I don't understand that argument. My point wasn't whether it's a good 
idea to write objects that compare unequal to themselves -- such objects 
already exist:

>>> nan = float("nan")
>>> nan == nan
False

I only warned that a list containing such an object does not meet the 
intuitive expectation that list_a == list_b implies that all items in list_a 
compare equal to the respective items in list_b.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Implementation of list comparison operators

[Peter Otten]

One obscure detail of the implementation of list equality:

In Python an object can be unequal to itself:

>>> class A:
... def __eq__(self, other): return False
... 
>>> a = A()
>>> a == a
False

However, the list assumes that (a is a) implies a == a, so

>>> [a] == [a]
True


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Implementation of list comparison operators, was Re: Doubt in Python

[Peter Otten]

Alan Gauld via Tutor wrote:

> On 17/01/2019 07:24, Maninath sahoo wrote:
>  a=[100,50,30]
>  b=[100,90,3]
>  a > True
>  a>b
> > False
> >
> >
>> How it compares between two lists
>>
> The algorithm is probably described somewhere in the documentation
> but my understanding is that it looks something like this(in pdeudo code):

> def isGreater(X,Y):
> 
>if len(X) <= len(Y) return False
> 
>if X[0] < Y[0] return False
> 
>if X[0] == Y[0] return X[1:] > Y[1:]
> 
>return True
> 
> 
> And similarly for the other operators.
> 
> But that's just based on playing with the interpreter and exploring
> different values...

>>> [10] > [1, 2, 3]
True
>>> [1] > [10, 2, 3]
False

So you cannot decide on length alone.

My analogy of list comparison would be words in a dictionary; you compare 
individual letters until you reach a decision:

mad <--> mad

m -> equal
a -> equal
d -> equal
(no character on both sides) --> mad == mad

madam <--> mad

m -> equal
a -> equal
d -> equal
a is greater than (no character) --> madam > mad

madam <--> man

m -> equal
a -> equal
d is less than n --> madam < man


A tentative implementation of less() (untested):

# zip() intentionally omitted

def less(x, y):
for i in range(min(len(x), len(y))):
if not x[i] == y[i]:
return x[i] < y[i]
return len(x) < len(y)

Finally here's a tool that wraps all comparison operations of the list 
items.

$ cat track_compare.py
class A:
def __init__(self, value):
self.value = value
def __lt__(self, other):
print(self, "?", other)
return self.value > other.value
def __le__(self, other):
print(self, "<=?", other)
return self.value <= other.value
def __ge__(self, other):
print(self, ">=?", other)
return self.value >= other.value
def __ne__(self, other):
print(self, "!=?", other)
return self.value != other.value
def __repr__(self):
return repr(self.value)


def noisy(items):
return [A(item) for item in items]

a = noisy([1, 2, 3])
b = noisy([1, 10, 20])
c = noisy([1])
d = noisy([1])

print("{} < {}".format(a, b))
a < b

print("\n{} != {}".format(c, d))
c != d

print("\n{} == {}".format(c, d))
c == d

$ python3 track_compare.py 
[1, 2, 3] < [1, 10, 20]
1 ==? 1
2 ==? 10
2 https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Debugging a sort error.

[Peter Otten]

mhysnm1...@gmail.com wrote:

> Issue, following error is generated after trying to sort a list of
> strings.
> 
> description.sort()
> TypeError: unorderable types: float() < str()

Consider

>>> descriptions = ["foo", "bar", 123, 3.14, 42, 200.1, "0"]
>>> sorted(descriptions)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unorderable types: int() < str()

If there are only numbers and strings in the list you can force the sort to 
succeed with the following custom key function:

>>> def key(item):
... return isinstance(item, str), item
... 

This will move the numbers to the beginning of the list:

>>> sorted(descriptions, key=key)
[3.14, 42, 123, 200.1, '0', 'bar', 'foo']


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Doubt

[Peter Otten]

Amit Yadav wrote:

> How can simply typing
> 
>  print "hello world"
> 
> work?
> Like without including any header file or import statements how can it
> work.

In Python 2 print is part of the syntax that the compiler knows, just like 

int

or

for (... ) {}

in C. In Python 3 print is just a name, like len, say, in both py2 and py3.
A name on the module level is looked up first in the module's namespace, and 
if it's not found in another special namespace that is built into the 
interpreter:

>>> __builtins__

>>> __builtins__.len is len
True

The unqualified name "len" is the same object as __builtins__.len because it 
is looked up there. When you overwrite it with your own name this will take 
precedence:

>>> def len(x): return 42
... 
>>> __builtins__.len is len
False
>>> len("abc")
42

OK that was probably a bad idea, let's remove our len() to see the built-in 
again:

>>> del len
>>> len("abc")
3

The built-in namespace is user-writeable:

>>> __builtins__.spam = "ham"
>>> spam
'ham'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Log file for Nested if-elif

[Peter Otten]

Asad wrote:

> Hi All ,
> 
> Need advice on the following piece of code :
> 
> with open(r"file1.log", 'r') as f:
> tail = deque(maxlen=8)  # the last eight lines
> script = None
> for line in f:
> tail.append(line)
> if
> re.search('\?/patch/\d{8}/\d{8}/admin/load.sql',line,re.IGNORECASE):
> script = line
> elif re.search(r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}',
> line, re.IGNORECASE):
> script = line
> elif re.search(r'set_metadata', line ,re.IGNORECASE) is not None:
> print "Reason of error \n", tail[-1]
> print "Script:\n", script
> print "Block of code:\n"
> for item in tail:
>  print item
> print " Danger "
> break
> Now this is printing the last cached line in the variable line   . However
> I would like to see the following output :
> 
> 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then
> 
> look for the line "set_metadata" in the file1.log  if it finds the pattern
> then print the line which matches the pattern
> \?/patch/\d{8}/\d{8}/admin/load.sql
> 
> print last 4 lines of the tail array  and exit
> 
> 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql'
> 
> then look of the anothern pattern
> :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern
> 
> then look for line "set_metadata" in the file1.log  if it finds the
> pattern then print the line which matches the pattern
> \?/patch/\d{8}/\d{8}/admin/load.sql
> 
> print all the lines in tail
> 
> print a recommendation "Please check the installation"
> 
> 
> 3 ) if it doesnot match  the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql
> or '\?/patch/\d{8}/\d{8}/admin/load.sql'
> 
> print "No match found refer to install guide"
> 
> Can you advice what I can do to change the code .

You have "nested" in the subject, do not make an attempt to structure your 
loop. Why is that?

In my experience using lots of small functions makes code easier to 
understand. Below is a suggestion how you might break up your code.

# untested; expect a few bugs!
def with_tail(items, maxlen):
tail = deque(maxlen=maxlen)

def gen_items():
for item in items:
tail.append(item)
yield item

return tail, gen_items()


def search_end_of_section(lines):
for line in lines:
if re.search(r'set_metadata', line, re.IGNORECASE) is not None:
break
else:
raise ValueError("Reached end of file...panic!")


def dump(script, tail, advice=None):
print "Reason of error \n", tail[-1]
print "Script:\n", script
print "Block of code:\n"
for item in tail[-limit:]:
print item
print " Danger "
if advice is not None:
print advice


RE_PATCH = re.compile(r'\?/patch/\d{8}/\d{8}/admin/load.sql', re.IGNORECASE)
RE_APPLY = re.compile(
r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', re.IGNORECASE
)

with open(r"file1.log", 'r') as f:
tail, lines = with_tail(f)
for line in lines:
if RE_PATCH.search(line) is not None:
search_end_of_section(lines)
dump(script=line, tail=tail[-4:])
break
elif RE_APPLY.search(line) is not None:
search_end_of_section(lines)
dump(
script=line, tail=tail,
advice="Please check the installation"
)
break


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Defining variable arguments in a function in python

[Peter Otten]

Avi Gross wrote:

> To spare others, 

Thank you for that.

> I sent Steven alone

'Tis well deserved ;)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Defining variable arguments in a function in python

[Peter Otten]

Karthik Bhat wrote:

> Hello,
> 
> I have the following piece of code. In this, I wanted to make use
> of the optional parameter given to 'a', i.e- '5', and not '1'
> 
> def fun_varargs(a=5, *numbers, **dict):
> print("Value of a is",a)
> 
> for i in numbers:
> print("Value of i is",i)
> 
> for i, j in dict.items():
> print("The value of i and j are:",i,j)
> 
> fun_varargs(1,2,3,4,5,6,7,8,9,10,Jack=111,John=222,Jimmy=333)
> 
> How do I make the tuple 'number'  contain the first element to be 1 and
> not 2?

One option is to change the function signature to

def fun_varargs(*numbers, a=5, **dict):
...

which turns `a` into a keyword-only argument.

>>> fun_varargs(1, 2, 3, foo="bar"):
Value of a is 5
Value of i is 1
Value of i is 2
Value of i is 3
The value of i and j are: foo bar

To override the default you have to specify a value like this:

>>> fun_varargs(1, 2, 3, foo="bar", a=42)
Value of a is 42
Value of i is 1
Value of i is 2
Value of i is 3
The value of i and j are: foo bar


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python script errors

[Peter Otten]

Ravi Kumar wrote:

> Hi,
> 
> I have developed a python script to get api calls for meraki
> clientlogevents Thanks for all the help previously  I am facing few errors
> such as
> 
> Json_string=r.json()
> 
> raw_decode
> raise JSONDecodeError("Expecting value", s, err.value) from None
> json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

> 
> '{0:20}   {1:30}   {2:16}   {3:18}   {4:10} {5:11} '.format(
> deviceserial, type, macaddress,occurredAt, details))
> TypeError: unsupported format string passed to NoneType.__format__
> 
> I know this occurs when the api response is nulls but how do I handle
> this?

Say x can be a string or None. 

>>> for x in ["a string", None]:
... "{:20}".format(x)
... 
'a string'
Traceback (most recent call last):
  File "", line 2, in 
TypeError: unsupported format string passed to NoneType.__format__

The string can be padded while None does not support that. The best solution 
is usually to replace None with a string that cannot occur normally, and 
then to use that instead of None. Example:

>>> for x in ["a string", None]:
... "{:20}".format("#NONE" if x is None else x)
... 
'a string'
'#NONE   '

Slightly more convenient: you can force string conversion before padding:

>>> for x in ["a string", None]:
... "{!s:20}".format(x)
... 
'a string'
'None'




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Peter Otten]

Steven D'Aprano wrote:

> [...]
>> In python 2.6 print statement work as print "Solution"
>> however after import collection I have to use print with
>> print("Solution") is this a known issue ?
> 
> As Peter says, you must have run
> 
> from __future__ import print_function
> 
> to see this behaviour. This has nothing to do with import collection.
> You can debug that for yourself by exiting the interactive interpreter,
> starting it up again, and trying to print before and after importing
> collection.

To be fair to Asad -- I sneaked in the __future__ import into my sample 
code. I did it to be able to write Python 3 code that would still run in his 
2.6 interpreter. 

In hindsight that was not a good idea as it can confuse someone who has 
never seen it, and the OP has yet to learn other more important things.

 


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] print statement vs print() function, was Re: Increase performance of the script

[Peter Otten]

Asad wrote:

> Hi All,
> 
>   I used your solution , however found a strange issue with deque
>   :
> 
> I am using python 2.6.6:

Consider switching to Python 3; my code as posted already works with that.
 
 import collections
 d = collections.deque('abcdefg')
 print 'Deque:', d
>   File "", line 1
> print 'Deque:', d
>  ^
> SyntaxError: invalid syntax

This is the effect of a

from __future__ import print_function

import. Once that is used print becomes a function and has to be invoked as 
such. 

If you don't want this remove the "from __future__ print_function" statement 
and remove the parentheses from all print-s (or at least those with multiple 
arguments).

Note that in default Python 2

print("foo", "bar")

works both with and without and parens, but the parens constitute a tuple. 
So

>>> print "foo", "bar"  # print statement with two args
foo bar
>>> print ("foo", "bar")  # print statement with a single tuple arg
('foo', 'bar')
>>> from __future__ import print_function
>>> print "foo", "bar"  # illegal, print() is now a function
  File "", line 1
print "foo", "bar"
  ^
SyntaxError: invalid syntax
>>> print("foo", "bar")  # invoke print() function with two args
foo bar


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Peter Otten]

Asad wrote:

> Hi All ,
> 
>   I have the following code to search for an error and prin the
> solution .
> 
> /A/B/file1.log size may vary from 5MB -5 GB
> 
> f4 = open (r" /A/B/file1.log  ", 'r' )
> string2=f4.readlines()

Do not read the complete file into memory. Read one line at a time and keep 
only those lines around that you may have to look at again.

> for i in range(len(string2)):
> position=i
> lastposition =position+1
> while True:
>  if re.search('Calling rdbms/admin',string2[lastposition]):
>   break
>  elif lastposition==len(string2)-1:
>   break
>  else:
>   lastposition += 1

You are trying to find a group of lines. The way you do it for a file of the 
structure

foo
bar
baz
end-of-group-1
ham
spam
end-of-group-2

you find the groups

foo
bar
baz
end-of-group-1

bar
baz
end-of-group-1

baz
end-of-group-1

ham
spam
end-of-group-2

spam
end-of-group-2

That looks like a lot of redundancy which you can probably avoid. But 
wait...


> errorcheck=string2[position:lastposition]
> for i in range ( len ( errorcheck ) ):
> if re.search ( r'"error(.)*13?"', errorcheck[i] ):
> print "Reason of error \n", errorcheck[i]
> print "script \n" , string2[position]
> print "block of code \n"
> print errorcheck[i-3]
> print errorcheck[i-2]
> print errorcheck[i-1]
> print errorcheck[i]
> print "Solution :\n"
> print "Verify the list of objects belonging to Database "
> break
> else:
> continue
> break

you throw away almost all the hard work to look for the line containing 
those four lines? It looks like you only need the 
"error...13" lines, the three lines that precede it and the last 
"Calling..." line occuring before the "error...13".

> The problem I am facing in performance issue it takes some minutes to
> print out the solution . Please advice if there can be performance
> enhancements to this script .

If you want to learn the Python way you should try hard to write your 
scripts without a single

for i in range(...):
...

loop. This style is usually the last resort, it may work for small datasets, 
but as soon as you have to deal with large files performance dives.
Even worse, these loops tend to make your code hard to debug.

Below is a suggestion for an implementation of what your code seems to be 
doing that only remembers the four recent lines and works with a single 
loop. If that saves you some time use that time to clean the scripts you 
have lying around from occurences of "for i in range(): ..." ;)


from __future__ import print_function

import re
import sys
from collections import deque


def show(prompt, *values):
print(prompt)
for value in values:
print(" {}".format(value.rstrip("\n")))


def process(filename):
tail = deque(maxlen=4)  # the last four lines
script = None
with open(filename) as instream:
for line in instream:
tail.append(line)
if "Calling rdbms/admin" in line:
script = line
elif re.search('"error(.)*13?"', line) is not None:
show("Reason of error:", tail[-1])
show("Script:", script)
show("Block of code:", *tail)
show(
"Solution",
"Verify the list of objects belonging to Database"
)
break


if __name__ == "__main__":
filename = sys.argv[1]
process(filename)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Finding the largest gap in tuple between two lists.

[Peter Otten]

Mats Wichmann wrote:

> 1. From an endpoint pair - say Buy/Levski and Sell/Olisar, you need the
> common items.  No point in looking further at items that can't both be
> bought at the source and sold at the destination.  Fortunately, Python
> sets are really good at this, but first you have to extract only the
> item names from your dictionary, since that comparison is based only on
> good names.  Something like this:
> 
> def common(curr, other):
> return set(curr.keys()).intersection(set(other.keys()))

As dict.keys() are very similar to a set you can also write 

curr.keys() & other.keys()

or

curr.keys() & other  # if you don't care about symmetry ;)

> So one way to tackle this might be to build a new dictionary containing
> the items and their price difference. Given our previous idea of having
> a set which contains the common items:
> 
> prices = {k: Selling_Olisar[k] - Buying_Levski[k] for k in possibles}

While a dict is almost always a good idea there is no need to build one if 
you're only interested in one entry. Instead feed the pairs directly to 
max():

selling = dict(Selling_Olisar)
buying = dict(Buying_Levski)
possibles = selling.keys() & buying

if possibles:
profit, good = max(
(selling[k] - buying[k], k) for k in possibles
)
if profit > 0:
print("Best good", good, "wins", profit, "per unit")
else:
print("Found nothing profitable.")
else:
print("Found nothing that can be sold at your destination")

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem using termcolor and colorama

[Peter Otten]

John Blower wrote:

> I try to create colour text by using these two  modules , only returns the
> colour code, no colour text. How to solve this problem.
> 
 import termcolor
 import colorama
 colorama.init()
 termcolor.cprint('Hello World','red')
> [31mHello World[0m

Start with one module. Does

>>> import colorama
>>> print(colorama.Fore.RED + "Hello" + colorama.Style.RESET_ALL)
Hello

print Hello in red?

If yes, how do

>>> colorama.Fore.RED + "Hello" + colorama.Style.RESET_ALL
'\x1b[31mHello\x1b[0m'

and

>>> termcolor.colored("Hello", "red")
'\x1b[31mHello\x1b[0m'

differ on your system?

If no, does

>>> from ctypes import windll
Traceback (most recent call last):
  File "", line 1, in 
ImportError: cannot import name 'windll'

raise an exception on your system? This is expected on my (Linux) system, 
but on Windows it probably means that something went wrong.
The pypi page says

"""
I’ve personally only tested it on Windows XP (CMD, Console2), Ubuntu (gnome-
terminal, xterm), and OS X.
"""

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] saveLine

[Peter Otten]

Avi Gross wrote:

> Alan and others have answered the questions posed and what I am asking now
> is to look at the function he proposed to keep track of the last five
> lines.
> 
> There is nothing wrong with it but I wonder what alternatives people would
> prefer. His code is made for exactly 5 lines to be buffered and is quite
> efficient. But what if you wanted N lines buffered, perhaps showing a
> smaller number of lines on some warnings or errors and the full N in other
> cases?

The standard library features collections.deque. With that:

buffer = collections.deque(maxlen=N)
save_line = buffer.append

This will start with an empty buffer. To preload the buffer:

buffer = collections.deque(itertools.repeat("", N), maxlen=N)

To print the buffer:

print_buffer = sys.stdout.writelines

or, more general:

def print_buffer(items, end=""):
for item in items:
print(item, end=end)

Also, for smallish N:

def print_buffer(items, end=""):
print(*items, sep=end)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regex for Filesystem path (Asad)

[Peter Otten]

Albert-Jan Roskam wrote:

> I was thinking it would also be possible to do (in Windows):
> import os.path
> os.path.sep = '/'
> os.path.normpath('c:\\beeh/foo\\bar/baz')
> 
> But alas, this still creates normalized windows-style paths.

If your input data has only forward slashes you can keep it that way with 
posixpath, i. e.

>>> import posixpath
>>> posixpath.join("foo", "bar")
'foo/bar'

should produce the same result on windows.
posixpath.normpath() cannot replace backward with forward slashes because 
the backslash is an ordinary character on Linux et. al.

Fun fact: scripts written with Windows in mind sometimes work on Linux, with 
the output going to unexpected places:

$ python3
Python 3.4.3 (default, Nov 28 2017, 16:41:13) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> with open("c:\\foo\\bar.txt", "w") as f: print("hi", file=f)
... 
>>> 
$ ls
c:\foo\bar.txt


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] best way to dynamically set class variables?

[Peter Otten]

Alan Gauld via Tutor wrote:

>>exec() might even be a way
> 
> It's a way but it's a bad way! :-)

And then

> A simple approach you could use would be to get Python to
> generate a new python file(module) containing the required class
> definition (simple string processing) and then dynamically
> import the new file.

That's basically exec(), with better tracebacks and a higher chance to run 
outdated code ;)

By the way I don't think exec() is bad as long as you control its input and 
as long as this input is fairly simple.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to print lines within two timestamp

[Peter Otten]

Alan Gauld via Tutor wrote:

> On 26/10/2018 18:45, Alan Gauld via Tutor wrote:
> 
>> It woiyukld
> 
> No idea what happened there. Should be "would" of course!

Of coiyukrse! Nobody thoiyukght otherwiiyske :)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What is the best way for a program suite to know where it is installed?

[Peter Otten]

eryk sun wrote:

> On 10/22/18, boB Stepp  wrote:
>>
>> Importing the various program modules/module contents is
>> no issue.  Where I believe I need to know the paths to things are to
>> get to data folders, config files, and occasionally utility programs
>> that I have written that are on my hard drive, but not copied to my
>> current program suite.
> 
> It's common to use __file__ for a portable application, except you
> should use sys.executable if it's a frozen executable, i.e. if
> sys.frozen exists.
> 
> For an installed application, limit this approach to the application's
> immutable resources. Don't use the installation directory to store
> modifiable data. You may not have write access (e.g. a system
> installation and the user lacks root/admin access, or it's an
> immutable directory or read-only disk mount).

I've not tried it, but for per-app read-only data importlib.resources
appears to be the right tool.

https://docs.python.org/dev/library/importlib.html#module-importlib.resources


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[Peter Otten]

Cameron Simpson wrote:

> On 21Oct2018 10:55, Peter Otten <__pete...@web.de> wrote:
>>boB Stepp wrote:
>>> So I am now wondering if using
>>> tempfile.NamedTemporaryFile(delete=False) would solve this problem
>>> nicely?  As I am not very familiar with this library, are there any
>>> unforeseen issues I should be made aware of?  Would this work equally
>>> well on all operating systems?
>>
>>I think this is cool thinking outside of the box.
>>
>>I would not have "dared" this, but now you suggest it I cannot see
>>anything wrong with your approach.
> 
> The doco for mktemp (do not use! use mkstemp or the NamedTemporaryFile
> classes instead!) explicitly mentions using delete=False.

Well, "permanent temporary file" does sound odd.

By the way, NamedTemporaryFile returns a proxy instead of the file itself. 
In some rare cases that could be a problem.

Would mktemp() really be dangerous if you used it like this,

def new_game(directory):
for _retry in range(3):
filename = mktemp("game_", ".json", dir=directory)
try:
   return open(filename, "x")
except FileExistsError:
   pass
raise FileExistsError


with the "x" mode?




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Some questions about importlib

[Peter Otten]

Quentin Agren wrote:

> Hi,
> 
> My name Quentin, and this is my first post to this list so please redirect
> me if this is not the proper audience.
> 
> I have been studying the 'importlib' standard library package this past
> week, and although I find it very readable I am puzzled by some questions
> that I'd like to share.
> 
> - [Chicken and egg] Does the 'import' statement in Python make use of
> 'importlib'? If so, how are the imports in 'importlib' itself carried out?
> (for example in __init__.py)

Google found

http://sayspy.blogspot.com/2012/02/how-i-bootstrapped-importlib.html

> - Why does 'importlib' use 'nt' or 'psoix' (see
> '_bootstrap_external._setup()') rather than the portable 'os', and
> reimplement some of the latter's functionality, for example '_path_join',
> or '_path_split'. Another example is in 'SourceFileLoader.set_data()'
> where the logic of 'os.mkdirs' is reproduced to create all necessary
> intermediary directories in a file path.
> 
> - Similarly, would not using the 'struct' module simplify the
> packing/unpacking of bytecode. For example by defining BYTECODE_HEADER_FMT
> = '4sII' (Python 3.6)

If the bootstrap process still works as Brett Cannon described it back in 
2012 struct and os cannot be used because they aren't built-in modules.

>>> import sys
>>> "posix" in sys.builtin_module_names
True
>>> "os" in sys.builtin_module_names
False
>>> "struct" in sys.builtin_module_names
False



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[Peter Otten]

boB Stepp wrote:

> Use case:  I want to allow a user of my Solitaire Scorekeeper program
> to be able to give any name he wants to each game of solitaire he
> wishes to record.  My thought for permanent storage of each game's
> parameters is to use a dictionary to map the user-chosen game names to
> a unique json filename.  This way I hope no shenanigans can occur with
> weird characters/non-printing characters.
> 
> My initial thought was to just have a sequence of game names with
> incrementing numerical suffixes:  game_0, game_1, ... , game_n.  But
> this would require the program to keep track of what the next
> available numerical suffix is.  Additionally, if a user chooses to
> delete a game, then there would be a gap in the numerical sequence of
> the game filenames.  I find such a gap aesthetically displeasing and
> would probably go to additional efforts to reuse such deleted
> filenames, so there would be no such "gaps".
> 
> So I am now wondering if using
> tempfile.NamedTemporaryFile(delete=False) would solve this problem
> nicely?  As I am not very familiar with this library, are there any
> unforeseen issues I should be made aware of?  Would this work equally
> well on all operating systems?

I think this is cool thinking outside of the box. 

I would not have "dared" this, but now you suggest it I cannot see anything 
wrong with your approach.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When are "__init__.py" files needed and not needed in a project?

[Peter Otten]

boB Stepp wrote:

> Linux Mint 19 Cinnamon, Python 3.6.6
> 
> I would have sworn that I had read, either on this list or the main
> Python list, that in the most recent versions of Python 3 that
> "__init__.py" files were no longer needed in nested project file
> structures.  

If you omit the __init__.py you get a "namespace package".

Namespace packages can combine multiple directories into one package, but 
will be shaded by "normal" packages.

> But when I attempted to run tests for the first time on
> my new Solitaire Scorekeeper project (Finally getting around to
> this!), I got:
> 
> bob@Dream-Machine1:~/Projects/solitaire_scorekeeper$ python3 -m unittest
> 
> --
> Ran 0 tests in 0.000s
> 
> OK
> 
> So no tests were run.  So it immediately occurred to me to add an
> empty "__init__.py" file to my "tests" subfolder and got what I was
> currently expecting:
> 
> bob@Dream-Machine1:~/Projects/solitaire_scorekeeper$ python3 -m unittest
> E
> ==
> ERROR: test_get_gamenames_bad_path
> (tests.tests_main.TestGameNamesMapperMethods) Test that when the method,
> get_gamenames(), is passed a path to a
> --
> Traceback (most recent call last):
>   File "/home/bob/Projects/solitaire_scorekeeper/tests/tests_main.py",
> line 20, in test_get_gamenames_bad_path
> self.assertEqual(gamenames.gamenames(), {})
> NameError: name 'self' is not defined
> 
> --
> Ran 1 test in 0.000s
> 
> FAILED (errors=1)
> 
> I was expecting this error and will shortly correct it.  So my
> question remains, when are "__init__.py" files needed and when are
> they not?

I am struggling with the differences between the two types of packages 
myself, so my first guess was that you accidentally found out that test
discovery doesn't work for namespace packages. However

https://docs.python.org/dev/library/unittest.html#test-discovery

expicitly states

"""
Changed in version 3.4: Test discovery supports namespace packages.
"""

Perhaps you should file a bug report.

> In case it helps, my current project structure is:
> 
> ~/Projects
> data/
> docs/
> tests/
> .git/
> main.py
> .gitignore
> 
> TIA!
> 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to find optimisations for code

[Peter Otten]

Steven D'Aprano wrote:

> We don't need to check that the individual letters are the same, because
> checking the counts will suffice. If they are not the same, one string
> will have (let's say) two A's while the other will have none, and the
> counts will be different.

Another great optimisation is solving the actual problem ;)

The OP isn't looking for anagrams, he wants to know if string2 can be spelt 
with the characters in string1:

abba, abacus --> True (don't care about the extra b, c, u, s)
abba, baab --> True (don't care about character order)
abba, bab --> False (bab is missing an a)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to find optimisations for code

[Peter Otten]

Pat Martin wrote:

> So should we always use sets or dictionaries if possible? Are these more
> efficient because of not keeping track of their position?

Not always. If you want to know how often one entry/char occurs in a linear 
storage like a list, a string, or a file, then you can loop over the data:

num_c = some_string.count(c)  # the loop is hidden in the count() method


pairs = (line.partition("=")[::2] for line in file)
value = next(value for key, value in pairs if key == wanted_key)

However, if you expect frequent similar questions go with a lookup table:

freq = Counter(some_string)
for c in strings.ascii_lowercase:
print(c, "occurs", freq[c], "times")


lookup = dict(pairs)
for key in wanted_keys:
print(key, "is", lookup.get(key, "unknown"))

Before you start measure if the effort may be worthwhile, i. e. whether the 
linear approach causes a relevant delay. 

When you're done doublecheck if you got the desired improvement, i. e. 
whether you optimised the right place.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to find optimisations for code

[Peter Otten]

Mats Wichmann wrote:

> On 10/19/2018 10:12 AM, Pat Martin wrote:
>> Sorry my first email didn't have a subject line
>> 
>> TLDR; How do you figure out if code is inefficient (if it isn't
>> necessarily obvious) and how do you find a more efficient solution?
> 
> I think you've hit it in your last sentence ("except maybe write more
> code and get more experience"): experience will let you recognize
> patterns.
> 
>> I use code wars sometimes to get some practice with Python, there was a
>> challenge to compare two strings and if string1 had enough characters to
>> be rearranged to make string2 return True, otherwise return False.
>> 
>> I wrote a script that was like this:
>> 
>> for i in string1:
>> if i not in string2:
>> return False
>> string2.replace(i,"",1)
>> return True
>> 
>> This worked but I kept getting that my function was too inefficient and
>> it took too long. I did a search for the problem and found someone was
>> using collections.Counter. This basically takes the string and returns
>> the number of times each character occurs in it. Then just compare the
>> count of one string to another to see if there is enough of each letter
>> to make the other string. This seems like an elegant way to do it.
> 
> notwithstanding that the challenge is a little contrived... here's
> something you will come to recognize.  You use the string replace
> method, which is described thus, pasting from the official docs:
> 
> """
> str.replace(old, new[, count])
> 
> Return a copy of the string with all occurrences of substring old
> replaced by new. If the optional argument count is given, only the first
> count occurrences are replaced.
> """
> 
> Strings are not modifiable, so there is no replace-in-place.  Each time
> you call string2.replace you build a new string... and then do nothing
> with that string, as you never assign a name to it. So that line clearly
> makes your code inefficient - you do some work that is just thrown away.
> And it's not clear what the purpose of this replacement is, anyway.

This is probably retyped from memory. If the line were

string2 = string2.replace(i,"",1)

it would address your concern below about repeated chars.

>>> def contains(s, t):
... for c in s:
... if c not in t: return False
... t = t.replace(c, "", 1)  # remove one occurence of c from t
... return True
... 
>>> contains("ata", "attax")
True
>>> contains("tata", "attax")
True
>>> contains("tatat", "attax")  # not enough 't's
False

> Further it's not clear that you have the right answer.  What if string1
> contains one 'r' and string2 contains three?  For the one 'r', the test
> that it is also in string2 succeeds... but nowhere do you find out that
> you actually needed to have three in order to be able to "rearrange to
> make string2".
> 
> Solving this problem might benefit from thinking about tests first: if
> you write a test where you know the answer is going to be negative, a
> second where it is going to be positive, and a third that is a
> non-trivial instance of positive (like the multiple-letter count), then
> you have something to code your solution against. After it works, you
> can then think about refactoring: is there a better way? This will kind
> of naturally lead you to thinking in terms of efficiency.
> 
> Lesson 2: for very many tasks, someone has already done it, and you can
> benefit by using some existing code, from the standard library or a
> module installed separately.  That's often the difference between a
> programming challenge, which may want you to code it yourself to show
> you can, and real-world problem solving, where you are rewarded (in
> time) by efficiently reusing existing code.
> 
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to find optimisations for code

[Peter Otten]

Pat Martin wrote:

> Sorry my first email didn't have a subject line
> 
> TLDR; How do you figure out if code is inefficient (if it isn't
> necessarily obvious) and how do you find a more efficient solution?
> 
> I use code wars sometimes to get some practice with Python, there was a
> challenge to compare two strings and if string1 had enough characters to
> be rearranged to make string2 return True, otherwise return False.
> 
> I wrote a script that was like this:
> 
> for i in string1:
> if i not in string2:
> return False
> string2.replace(i,"",1)
> return True
> 
> This worked but I kept getting that my function was too inefficient and it
> took too long. I did a search for the problem and found someone was using
> collections.Counter. This basically takes the string and returns the
> number of times each character occurs in it. Then just compare the count
> of one string to another to see if there is enough of each letter to make
> the other string. This seems like an elegant way to do it.
> 
> My question is, how do I know something is inefficient and more
> importantly how do I go about finding a more efficient solution?
> 
> I have just discovered dir() and it has really helped in finding methods
> for items but that doesn't help when finding actual packages especially if
> I don't know exactly what I am looking for.
> 
> Sorry about the long email, not sure if there is even an answer to this
> question except maybe write more code and get more experience?

In Python my rule of thumb is: replace lists and loops with dict and set 
lookups whenever possible. One small challenge with this pattern is that not 
all loops look like a loop:

char in some_string  # "slow"; have to loop through the whole string
char in set_of_chars # "fast"; calculate some hash value and then likely
 # jump to the right set entry



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] problem with creating paths

[Peter Otten]

Shall, Sydney via Tutor wrote:

> There are two items that are 'wrong' in this output.
> 
> 1. The property 'paths' is defined in the program as a list and the
> items are added using paths.append(), yet the test says that when tested
> it is a tuple.

>>> paths = ["foo", "bar"],
>>> paths += "baz",  # ("baz",) leads to the same result
>>> paths
(['foo', 'bar'], 'baz')

This might happen if you have accidental trailing commas in two places, but 
that seems rather unlikely. Maybe you have a function with a star

def f(*args):
... # args is a tuple, even if you pass one argument or no arguments
# to f().

?

> 2. The tuple arises by the addition of the last entry in the file, AFTER
> the closing bracket of the list which is the first item in the tuple.

Sorry, I don't understand that sentence.

> When I test the length of 'paths' I get a value of 2!

That's because you have a tuple with two entries, one list and one string.

> I apologise for the lengthy explanation, but I am at a loss.
> 
> I have looked for an error that might have added an item as a + and I
> find nothing.
> The character of the final item is also puzzling to me.
> 
> I would much appreciate any guidance as to how I should search for the
> fault or error.

Look at the assignments. If you find the pattern I mentioned above remove 
the commas. 

If you don't see anything suspicious make a test script containing only the 
manipulation of the paths variable. Make sure it replicates the behaviour of 
the bigger script; then show it to us.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python game error

[Peter Otten]

Alan Gauld via Tutor wrote:

> On 15/10/18 08:57, Peter Otten wrote:
> 
>> By the way, you do not need a map (dict) at all to implement a game like
>> this, you may return the next scene directly. A sketch:
>> 
>> class Bridge:
>> def enter(self):
>> ...
>> action = ...
>> if action == "jump off the bridge":
>> return Death("You are eaten by the piranhas")
>> else:
>> ...
> 
> That was my initial thought when I saw this but thee is one
> caveat. The original design creates a single instance of a
> scene and returns that on each access. The suggestion above
> creates a new instance on every call. So if it is important
> to use the same instance each time then the map is a
> better solution.
> 
> (Although you could instead create a class variable holding
> the first instance of itself then use a class  constructor
> to either create the instance or access the class variable...)
> 

Or, to keep it really simple, return global class instances:

DEATH_BY_TORNADO = Death("The storm swept you off the bridge")

>> class Bridge:
>> def enter(self):
>> ...
>> action = ...
   if action == "tornado":
   return DEATH_BY_TORNADO

One advantage of both variants is that tools like pylint are likely to catch 
spelling errors like the one that prompted the initial question.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python game error

[Peter Otten]

bob gailer wrote:


>  Python coding "trick"1  
> when I build a map I omit the () after the class e.g. 'death' = Death, 
> ... and apply them to the item retrieved from the map.
> 
> use a decorator to build the map dictionary:
> 
> # the decorator function:
> def add_to_map(cls, map={}): # map is initialized to a {} when function 
> is "compiled"
>  if cls:
>  map[cls.__name__] = cls # add entry to dictionary
>  return cls
>  else: return map
> 
> # apply decorator to class definitions
> # this will add 'Death': 
> @add_to_map
> class Death(Scene):
>  class_definition
> 
>   # ditto for all other classes based on Scene - then
> 
> class Map:
>  scenes  = add_to_map() # note when map is called with no arguments 
> it returns the dictionary

Hm, you have now replaced the dead simple

> class Map(object):
> scenes = {
> 'central_corridor': CentralCorridor(),
> 'laser_weapon_armory': LaserWeaponArmory(),
> 'the_bridge': TheBridge(),
> 'escape_pod': EscapePod(),
> 'death': Death()
> }

with a class decorator, a dunder name, a mutable default argument -- lots of 
clever Python specific stuff that I wouldn't put on a beginner's list of top 
priorities.

Don't use unless you spotted at least one bug in the code sample at first 
glance :)

By the way, you do not need a map (dict) at all to implement a game like 
this, you may return the next scene directly. A sketch:

class Bridge:
def enter(self):
...
action = ...
if action == "jump off the bridge":
return Death("You are eaten by the piranhas")
else:
...



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python game error

[Peter Otten]

Mariam Haji wrote:

> Hi guys,
> 
> I am still on the learn python the hard way.
> I built the below on python 2.7 and my OS is windows 7.
> 
> I am getting this error:
> Traceback (most recent call last):
>   File "ex43.py", line 205, in 
> a_game.play()
>   File "ex43.py", line 21, in play
> next_scene_name = current_scene.enter()
> AttributeError: 'NoneType' object has no attribute 'enter'

The value of current_scene is None here. If you dig a bit in your code 
looking for places where that can happen you end up with

> class Map(object):
> scenes = {
> 'central_corridor': CentralCorridor(),
> 'laser_weapon_armory': LaserWeaponArmory(),
> 'the_bridge': TheBridge(),
> 'escape_pod': EscapePod(),
> 'death': Death()
> }
> 
> def __init__(self, start_scene):
> self.start_scene = start_scene
> 
> def next_scene(self, scene_name):
> return Map.scenes.get(scene_name)

Map.scenes is a dict, and its get() method returns None when the lookup for 
a key fails. If you pass an unknown scene name to the next_scene() method it 
returns None rather than a Scene instance.

As a first step to fix this I recommend that you use the item lookup 
operator instead of the method:

  def next_scene(self, scene_name):
  return Map.scenes[scene_name]

This will fail immediately, and the error message should be more 
informative. I got:

Traceback (most recent call last):
  File "./game_mariam.py", line 181, in 
a_game.play()
  File "./game_mariam.py", line 20, in play
current_scene = self.scene_map.next_scene(next_scene_name)
  File "./game_mariam.py", line 173, in next_scene
return Map.scenes[scene_name]
KeyError: 'laser_weapon_armoury'

That is, you need to decide if you want British or American orthography.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] help please

[Peter Otten]

Michael Schmitt wrote:

> To whom it may concern:
> 
> 
> I am trying to teach myself Python and ran into a problem. This is my code

> I am getting the following error
>  for rivers in rivers.values():
> AttributeError: 'str' object has no attribute 'values'

> # prints river name
> for rivers in rivers.keys():
> print (rivers)

Look closely at the loop above. What is the value of the "rivers" variable 
after the first iteration?

To resolve the problem simply use a different name as the loop variable.

Pro tip: When you loop over a dict you get the keys by default. So:

for river in rivers:
print(river)

> # prints statement " The (river) is in the country of (country)
> for rivers in rivers:
> print ("The " + rivers.keys() + "is in the country of " + 
rivers.vaules())
> 

Here's a logic (and a spelling) error: rivers.keys() comprises all rivers 
and rivers.values() all countries in the dict. You want to loop over 
rivers.items() which gives you (river, country) pairs.

Tip: print() automatically inserts spaces between its arguments. So:

for river, country in rivers.items():
print("River", river, "is in", country)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Peter Otten]

Chip Wachob wrote:

> All,
> 
> Sorry for not being more clear.  I think I was nearing my fill of
> Python for the day when I wrote.
> 
> Now, refreshed, let me see if I can fill in the blanks a bit more.

You have posted code sprinkled with prints and irrelevant details.
Please rewrite the relevant part into a simple function, 

def bitshift_right(data, nshift):
   ...

just the pure algorithm without any prints. Then find input data that
produces an AssertionError in the snippet below:

input = ...
rshift = ...
wanted_output = ...
assert bitshift_right(input, rshift) == wanted_output

I'm sure then someone is able and willing to point out the error in your 
implementation.

In the mean time here's my effort which also provides an idea on how to wrap 
the shifting into a simple class:

$ cat bitshift_oo.py   
format_bits = "{:08b}".format
try:
format_bits(42)
except ValueError:
# python 2
def format_bits(b):
return bin(b)[2:].zfill(8)


def to_bits(s, sep=""):
return sep.join(map(format_bits, s))


def bitshift_right(b, n):
nbytes, nbits = divmod(n, 8)
length = len(b)
if nbits:
nbits = 8 - nbits
a = [0]
for bb in reversed(b):
hi, lo = divmod(bb << nbits, 256)
a[-1] |= lo
a.append(hi)
b = bytearray(a[::-1])
return (bytearray(nbytes) + b)[:length]


class Bytes(bytearray):
def __rshift__(self, n):
return Bytes(bitshift_right(self, n))

def __str__(self):
return to_bits(self, " ")


if __name__ == "__main__":
SAMPLE = Bytes(b"\xaa\xbb")
for i in range(12):
print(SAMPLE >> i)
$ python bitshift_oo.py
10101010 10111011
01010101 01011101
00101010 10101110
00010101 01010111
1010 10101011
0101 01010101
0010 10101010
0001 01010101
 10101010
 01010101
 00101010
 00010101
$

PS: An easier approach would be to match the to_bits() function with a 
to_bytes() function. Then you can implement the shift as a simple string 
manipulation:

def bitshift_right(b, n):
bits = to_bits(b)
shifted = ("0" * n + bits)[:len(bits)]
return to_bytes(shifted)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] guess my number game (reversed)

[Peter Otten]

chiara pascucci wrote:

> Hi,
> 
> sorry for "resurrecting" this thread. I have tried doing as suggested but
> with no luck.
> I now managed to make the programme work "partially". It works as desired
> for "too high" and "right" answer input, but when the users input "too
> low" the programme closes.
> here is what the code looks like right now.
> 
> print ("Think of a number from 1 to 100, and I will try to guess it")
> input ("press enter when ready...")
> 
> import random
> 
> number = random.randint(1,100)
> print (number)
> 
> answer = input ("right, too low, too high?")
> 
> while answer != "right":
> if answer == "too high":
> number2 = random.randint (1,number)
> print (number2)
> 
> elif answer == "too low":
> number3 = random.randit (number,100)
> print (number3)
> 
> answer = input ("is this right, too high, or too low?")
> 
> print ("I finally guessed your number")
> 
> input ("\n\n press the enter key to exit")
> 
> any help is really appreciated :)
> 
> Chiara

When you  run your script from the command line it not just terminates, it 
also prints an error message and important debug information (called 
"traceback") that helps you fix the code:

$ python3 guess_number_chiara.py 
Think of a number from 1 to 100, and I will try to guess it
press enter when ready...
63
right, too low, too high?too low
Traceback (most recent call last):
  File "guess_number_chiara.py", line 17, in 
number3 = random.randit (number,100)
AttributeError: 'module' object has no attribute 'randit'

The problem is in line 17 and there's a module that doesn't have a randit 
attribute. The only module mentioned in the line is random which indeed does 
not contain a randit() function, the actual function is named randint().

So the bug turns out to be a misspelt function name.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'

[Peter Otten]

Chip Wachob wrote:

> Hello,
> 
> I was not able to find any answers in the archive on this one.
> 
> I'm wondering if this task can be done in a better way than what I've
> attempted..
> 
> I have an array of bytes.  Up to 64, which makes for 512 bits.
> 
> I am reading these bytes in serially, and once I have a collection of
> them, I want to shift them by 'n' bits.  The size of the array and the
> number of bits are both variable up to the limit of 64/512.
> 
> Now, I've played around with looping through the bytes and taking the
> LSByte and shifting it by 'n' bits using >> or << and that works for
> the first byte.  But then I need to take the next byte in the sequence
> and shift it in the opposite direction by 8-n bits using << or >>
> (opposite the LSByte direction), respectively.  Then I OR the two
> bytes and save them into the location of the LSByte and then move to
> the next byte in the sequence and so on.  While this works most of the
> time, I sometimes get strange behavior at the 'fringes' of the bytes.
> Sometimes I end up with zero, or the shift seems to 'roll over'.
> 
> I'm thinking that maybe there's a way to treat the array / list and
> shift allowing the bits to transfer from byte to byte as needed.
> Maybe by concatenating the bytes into one huge word and then breaking
> it apart again?
> 
> I'm thinking that you folks out there know of a built-in function, or,
> an easier and more predictable way to accomplish the same.

Here are two ways to implement the left shift:

def bitshift(b, n, byteorder="big"):
size = len(b) + (n + 7) // 8
shifted = int.from_bytes(b, byteorder) << n
return shifted.to_bytes(size, byteorder)


def bitshift2(b, n):
nbytes, nbits = divmod(n, 8)
if nbits:
a = [0]
for bb in b:
hi, lo = divmod(bb << nbits, 256)
a[-1] |= hi
a.append(lo)
b = bytes(a)
return b + b"\x00" * nbytes


assert bitshift(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00"
assert bitshift2(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00"


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: How to roughly associate the values of two numpy arrays, or python lists if necessary

[Peter Otten]

Peter Otten wrote:

> Maybe you could sort the already-sorted property_b again, with some random
> offset:
> 
>>>> import itertools
>>>> def wiggled(items, sigma):
> ... counter = itertools.count()
> ... def key(item): return random.gauss(next(counter), sigma)
> ... return sorted(items, key=key)
> ...

One more example:

>>> s = """\
... But my actual scientific problem requires that the correlation should be 
... only approximate and I do not know how close to to a perfect correlation 
... it should be. So, I need to introduce some lack of good correlation when 
... I set up the correlation. How to do that is my problem.
... """
>>> print(textwrap.fill(" ".join(wiggled(s.split(), 2
But actual my scientific the requires that problem should only
correlation approximate be and not do I know how close to a perfect to
correlation should it So, be. I to lack need some introduce
correlation I of good when set correlation. up How to the that do
problem. is my

:)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: How to roughly associate the values of two numpy arrays, or python lists if necessary

[Peter Otten]

Shall, Sydney via Tutor wrote:

> What I want is the following.
> 
> I have:
> > property_a = [1, 6, 2, 4]
> > property_b = [62, 73, 31 102]
> 
> Result should approximately be:
> > property_b = [31, 102, 62, 73]
> 
> That is both lists change in value in exactly the same order.
> 
> Now, this is easy to achieve. I could simply sort both lists is
> ascending order and I would then have an exact alignment of values is
> ascending order. The correlation would be a perfect linear relationship,
> I suppose.
> 
> But my actual scientific problem requires that the correlation should be
> only approximate and I do not know how close to to a perfect correlation
> it should be. So, I need to introduce some lack of good correlation when
> I set up the correlation. How to do that is my problem.
> 
> I hope this helps to clarify what my problem is.

Maybe you could sort the already-sorted property_b again, with some random 
offset:

>>> import itertools
>>> def wiggled(items, sigma):
... counter = itertools.count()
... def key(item): return random.gauss(next(counter), sigma)
... return sorted(items, key=key)
... 
>>> wiggled(range(20), 3)
[0, 5, 2, 4, 1, 6, 7, 8, 3, 9, 11, 10, 13, 14, 16, 12, 18, 17, 19, 15]
>>> wiggled([31, 102, 62, 73], .8)
[102, 31, 62, 73]
>>> wiggled([31, 102, 62, 73], .8)
[31, 102, 62, 73]
>>> wiggled([31, 102, 62, 73], .8)
[31, 102, 62, 73]
>>> wiggled([31, 102, 62, 73], .8)
[31, 62, 102, 73]


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help understanding base64 decoding

[Peter Otten]

Ryan Smith wrote:

> Hello All,
> 
> I am currently working on a small utility that finds any base64
> encoded strings in files and decodes them. I am having issue
> understanding how the Base64 module actually works. The regular
> expression that I am using correctly matches on the encoded strings. I
> simply want to be able to convert the match of the encoded ascii
> string to it's decoded ascii equivalent. For example the base64
> encoded ascii string 'UwB5AHMAdABlAG0ALgBkAGwAbAA=' will decode to
> 'System.dll' if I use an online base64 decoder. However I get a
> completely different output when trying to codify this using python
> 3.6.5:
> 
import base64
import binascii
> 
test_str = 'UwB5AHMAdABlAG0ALgBkAGwAbAA='
 base64.b64decode(test_str)
> b'S\x00y\x00s\x00t\x00e\x00m\x00.\x00d\x00l\x00l\x00'
> 
temp = base64.b64decode(test_str)
binascii.b2a_base64(temp)
> b'UwB5AHMAdABlAG0ALgBkAGwAbAA=\n'
> 
> I understand that when decoding and encoding you have to use bytes
> objects but what I don't understand is why I can't get the proper
> conversion of the original ascii string. Can someone please point me
> in the right direction?

Look closely at the odd bytes in 

> b'S\x00y\x00s\x00t\x00e\x00m\x00.\x00d\x00l\x00l\x00'

or just do

>>> b'S\x00y\x00s\x00t\x00e\x00m\x00.\x00d\x00l\x00l\x00'[::2]
b'System.dll'

The even bytes are all NUL:

>>> b'S\x00y\x00s\x00t\x00e\x00m\x00.\x00d\x00l\x00l\x00'[1::2]
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

This means that your byte string already *is* the original string, encoded 
as UTF-16. You can convert it into a string with

>>> b'S\x00y\x00s\x00t\x00e\x00m\x00.\x00d\x00l\x00l\x00'.decode("utf-16")
'System.dll'

which will handle non-ascii characters correctly, too.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with building bytearray arrays

[Peter Otten]

Chip Wachob wrote:

> Peter,
> 
> I see that clue "[[".
> 
> The thread history pretty much sums up what is going on up to this point.
> 
> I'll cover it once more:

[snip]

> I hope this helps.

Unfortunately it doesn't as the problem is in my_transfer.
 
> I'm beginning to wonder if Python was the right choice for this
> project.. 

You mean you'd rather debug a segfault in C than an exception with a 
traceback and an error message that is spot on in Python?

You have an error in your program logic, and you'll eventually run into 
those no matter what language you choose.
 
> Thanks to everyone for your comments and patience.

If you are too frustrated to look into the actual problem at the moment you 
can change the following loop

> Earlier discussions here indicated that the best way was to :
> 
> results = []
> 
> for i in range (0, slice_size):
>results.append(transfer(data_out))

by replacing the append() with the extend() method

results = []
for i in range (0, slice_size):
results.extend(transfer(data_out))

for now.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with building bytearray arrays

[Peter Otten]

Chip Wachob wrote:

> Cameron,
> 
> Thank you again for the insight.
> 
> Yes, data_out is an equivalently-sized 'chunk' of a larger array.
> 
> I'm 'getting' this now..
> 
> So, without all the fluff associated with wiggling lines, my function
> now looks like this:
> 
> def RSI_size_the_loop():
>results = []
>all_together = []   # not certain if I need this, put it in in an
> attempt to fix the incompatibility if it existed
> 
>for x in range (0, MAX_LOOP_COUNT, slice_size):
>   results.append(my_transfer(disp, data_out, slice_size)
> 
>   print " results ", x, " = ", results  # show how results grows
> on each iteration
> 
>all_together = bytearray().join(results)
> 
>print " all together ", all_together
> 
> 
> I can observe results increasing in size and the last time through the
> loop:
> 
>  results  48  =
> 
[[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],

Note that there are two '[' at the start of the list. This means that the 
first list item is another list. In fact you seem to have a list of single 
item lists like

[["foo"], ["bar"], ...]

when you need

["foo", "bar", ...]

Of course join will fail with that:

>>> "".join(["foo", "bar"])
'foobar'
>>> "".join([["foo"], ["bar"]])
Traceback (most recent call last):
  File "", line 1, in 
TypeError: sequence item 0: expected string, list found

I think the error message is pretty clear ;)

Have a look into your my_transfer() function to find out why it returns a 
list (or show us the code).

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Writing for loop output to csv

[Peter Otten]

Cameron Simpson wrote:

> On 06Sep2018 12:32, Brandon Creech  wrote:
>>Hi, I am working to forecast the temperatures for the next 5 days using an
>>API and a for loop. I would like write the output of this loop to a csv in
>>this format::
>>
>>Columns: City, min1, max1, min2, max2,min3,max3,min4,max4,min5,max5
>>data: Athens,Greece 25.4,26.7etc.
>> Nantou,Taiwan 16.18, ..etc

> print call for the output.  The csv module provides a writer object which
> does all the heavy lifting for you: turns strings into quotes strings,
> puts in commas, etc. Try this change:
> 
>   csvw = csv.writer(sys.stdout)
>   for city, coords in cities.items():
>   weather = ForecastIO.ForecastIO( api_key, latitude=coords[0],
>   longitude=coords[1] ) 
>   daily = FIODaily.FIODaily(weather)
>   for day in range(2,7):
>   day_data = daily.get_day(day)
>   csvw.writerow([city, day_data['temperatureMax'],
>   day_data['temperatureMin']])

If you want all min/max pairs in the same row you can prebuild that row in a 
list. With minimal changes to Cameron's code:

# untested
csvw = csv.writer(sys.stdout)
for city, coords in cities.items():
weather = ForecastIO.ForecastIO(
api_key, latitude=coords[0], longitude=coords[1]
)
daily = FIODaily.FIODaily(weather)
row = [city]
for day in range(2, 7):
day_data = daily.get_day(day)
row.append(day_data['temperatureMin'])
row.append(day_data['temperatureMax'])
csvw.writerow(row)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A Python program to Summarise weather results?

[Peter Otten]

Matthew Polack wrote:

> Hi,
> 
> We have a spreadsheet from a local weather station.it summarises the
> amount of rainfall per day since 1863.
> 
> 
http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_nccObsCode=136_display_type=dailyDataFile_startYear=1863_c=-1249186659_stn_num=079028
> 
> We'd love to be able to summarise this into years eg. Total rainfall of
> 1863, 1864 all the way through to 2018.
> 
> Would Python be able to do this using the csv file?

Yes. 

Python has powerful tools to make this concise, like csv.DictReader() and 
itertools.groupby(), or -- even more highlevel -- pandas.

But you want to learn Python, don't you? 

Therefore I recommend that you try to solve this with just open() and a for 
loop to read the lines, and the str.split() method to break the lines into 
fields. With 'if ' you check whether the year has changed so that you 
need to print the current total and reset the total rainfall.

Give it a try, see how far you get, and we'll help you over the obstacles 
once you can present some rudimentary code.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Problem compiling code from GitHub

[Peter Otten]

Dave Hill wrote:

> I did as suggested but with the same result.

Make sure that your script is *not* in the same directory as ODSWriter.py 
and that the directory containing ODSWriter is *not* in sys.path. 

Then try again.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] need help generating table of contents

[Peter Otten]

Albert-Jan Roskam wrote:

> 
> From: Tutor  on behalf
> of Peter Otten <__pete...@web.de> Sent: Friday, August 24, 2018 3:55 PM
> To: tutor@python.org
> 
>> The following reshuffle of your code seems to work:
>> 
>> print('\r\n** Table of contents\r\n')
>> pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'
>> 
>> def process(triples, limit=None, indent=0):
>> for index, (title, page, count) in enumerate(triples, 1):
>> title = indent * 4 * ' ' + title
>> print(title.ljust(79, ".") + page.zfill(2))
>> if count:
>> process(triples, limit=int(count), indent=indent+1)
>> if limit is not None and limit == index:
>>  break
>> 
>> process(iter(re.findall(pattern, toc, re.DOTALL)))
> 
> Hi Peter, Cameron,
> 
> Thanks for your replies! The code above indeeed works as intended, but: I
> don't really understand *why*. I would assign a name to the following line
> "if limit is not None and limit == index", what would be the most
> descriptive name? I often use "is_*" names for boolean variables. Would
> "is_deepest_nesting_level" be a good name?

No, it's not necessarily the deepest level. Every subsection eventually ends 
at this point; so you might call it

reached_end_of_current_section

Or just 'limit' ;) 

The None is only there for the outermost level where no /Count is provided. 
In this case the loop is exhausted.

If you find it is easier to understand you can calculate the outer count aka 
limit as the number of matches - sum of counts:

def process(triples, section_length, indent=0):
for index, (title, page, count) in enumerate(triples, 1):
title = indent * 4 * ' ' + title
print(title.ljust(79, ".") + page.zfill(2))
if count:
process(triples, section_length=int(count), indent=indent+1)
if section_length == index:
break

triples = re.findall(pattern, toc, re.DOTALL)
toplevel_section_length = (
len(triples)
- sum(int(c or 0) for t, p, c in triples)
)
process(iter(triples), toplevel_section_length)

Just for fun here's one last variant that does away with the break -- and 
thus the naming issue -- completely:

def process(triples, limit=None, indent=0):
for title, page, count in itertools.islice(triples, limit):
title = indent * 4 * ' ' + title
print(title.ljust(79, ".") + page.zfill(2))
if count:
process(triples, limit=int(count), indent=indent+1)

Note that islice(items, None) does the right thing:

>>> list(islice("abc", None))
['a', 'b', 'c']


> Also, I don't understand why iter() is required here, and why finditer()
> is not an alternative.

finditer() would actually work -- I didn't use it because I wanted to make 
as few changes as possible to your code. What does not work is a list like 
the result of findall(). This is because the inner for loops (i. e. the ones 
in the nested calls of process) are supposed to continue the iteration 
instead of restarting it. A simple example to illustrate the difference:

 >>> s = "abcdefg"
>>> for k in range(3):
... print("===", k, "===")
... for i, v in enumerate(s):
... print(v)
... if i == 2: break
... 
=== 0 ===
a
b
c
=== 1 ===
a
b
c
=== 2 ===
a
b
c
>>> s = iter("abcdefg")
>>> for k in range(3):
... print("===", k, "===")
... for i, v in enumerate(s):
... print(v)
... if i == 2: break
... 
=== 0 ===
a
b
c
=== 1 ===
d
e
f
=== 2 ===
g




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] need help generating table of contents

[Peter Otten]

Albert-Jan Roskam wrote:

> Hello,
> 
> I have Ghostscript files with a table of contents (toc) and I would like 
to use this info to generate a human-readable toc. The problem is: I can't 
get the (nested) hierarchy right.
> 
> import re
> 
> toc = """\
> [ /PageMode /UseOutlines
>   /Page 1
>   /View [/XYZ null null 0]
>   /DOCVIEW pdfmark
> [ /Title (Title page)
>   /Page 1
>   /View [/XYZ null null 0]
>   /OUT pdfmark
> [ /Title (Document information)
>   /Page 2
>   /View [/XYZ null null 0]
>   /OUT pdfmark
> [ /Title (Blah)
>   /Page 3
>   /View [/XYZ null null 0]
>   /OUT pdfmark
> [ /Title (Appendix)
>   /Page 16
>   /Count 4
>   /View [/XYZ null null 0]
>   /OUT pdfmark
> [ /Title (Sub1)
>   /Page 17
>   /Count 4
>   /OUT pdfmark
> [ /Title (Subsub1)
>   /Page 17
>   /OUT pdfmark
> [ /Title (Subsub2)
>   /Page 18
>   /OUT pdfmark
> [ /Title (Subsub3)
>   /Page 29
>   /OUT pdfmark
> [ /Title (Subsub4)
>   /Page 37
>   /OUT pdfmark
> [ /Title (Sub2)
>   /Page 40
>   /OUT pdfmark
> [ /Title (Sub3)
>   /Page 49
>   /OUT pdfmark
> [ /Title (Sub4)
>   /Page 56
>   /OUT pdfmark
> """
> print('\r\n** Table of contents\r\n')
> pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'
> indent = 0
> start = True
> for title, page, count in re.findall(pattern, toc, re.DOTALL):
> title = (indent * ' ') + title
> count = int(count or 0)
> print(title.ljust(79, ".") + page.zfill(2))
> if count:
> count -= 1
> start = True
> if count and start:
> indent += 2
> start = False
> if not count and not start:
> indent -= 2
> start = True
> 
> This generates the following TOC, with subsub2 to subsub4 dedented one 
level too much:

> What is the best approach to do this?
 
The best approach is probably to use some tool/library that understands 
postscript. However, your immediate problem is that when there is more than 
one level of indentation you only keep track of the "count" of the innermost 
level. You can either use a list of counts or use recursion and rely on the 
stack to remember the counts of the outer levels.

The following reshuffle of your code seems to work:

print('\r\n** Table of contents\r\n')
pattern = '/Title \((.+?)\).+?/Page ([0-9]+)(?:\s+/Count ([0-9]+))?'

def process(triples, limit=None, indent=0):
for index, (title, page, count) in enumerate(triples, 1):
title = indent * 4 * ' ' + title
print(title.ljust(79, ".") + page.zfill(2))
if count:
process(triples, limit=int(count), indent=indent+1)
if limit is not None and limit == index:
break

process(iter(re.findall(pattern, toc, re.DOTALL)))



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Times Tables Program that constantly tells you that you are wrong!

[Peter Otten]

Matthew Polack wrote:

> I'm also trying to solve the rounding issue...but can't work out the
> syntax using the example provided...have tried this but I get an error...
> 
> _tkinter.TclError: unknown option "-text %.2f"
> 
> .I'm sure it is a simple syntax issue...but I don't know what it is.
> 
> def viewPercent():
>  percentCalc = score/total*100
>  percentViewLab["text %.2f"] % percentCalc

I'm sure you can resolve that one yourself. You have

obj[key] = float_value

Now you want to format the float value as a string showing a decimal number.
So all you need to change is the righthand side. If it helps write it in two 
lines:

some_string = ... expression involving float_value ...
obj[key] = some_string

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Times Tables Program that constantly tells you that you are wrong!

[Peter Otten]

Matthew Polack wrote:

> I'm working my way through some of the tips you provided and tried to use
> the code givenbut am getting an error at Line 75 in my code re:  not
> enough arguments for format string
> 
> _
> return self.func(*args)
>   File "Timespicture.py", line 75, in checkanswer
> Your current score is: %f""" % answer,score
> TypeError: not enough arguments for format string

> fail_str = """
> Sorry, you got it wrong,
> the correct answer was %d
> Your current score is: %f""" % answer,score

Python reads 

some_str % arg1, arg2

as

(some_str % arg1), arg2

To make the above work you have to use parens:

fail_str = """
Sorry, you got it wrong,
the correct answer was %d
Your current score is: %f""" % (answer, score)

Another option is to use f-strings to sidestep the issue:

fail_str = f"""
Sorry, you got it wrong,
the correct answer was {answer}
Your current score is: {score}"""


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Times Tables Program that constantly tells you that you are wrong!

[Peter Otten]

Matthew Polack wrote:

> Hi All,
> 
> Thanks to your help I've nearly got my demo 'Times Tables' program fully
> working.
> 
> The last remaining issue is the program does not calculate the percentage
> right when you make mistakes...it just keeps giving a result of 100%.
> 
> I've looked and head scratched...and am not sure.
> 
> Can anyone figure out what is wrong with this code?

You increment the score no matter whether the answer is correct or not.

> if response == answer:
> score = int(score + 1)

> else :
> score= int(score + 1)

Random remarks

- Your code suffers from "over-globalisation". You only need to declare a 
variable global when you want to change it from within a function. Example:

x = 0

def show_x():
   # no global necessary
   print("x =", x)

def inc_x():
global x
x += 1

- You create a new Label in every call of viewXXX(). For long-running 
applications that will consume a lot of memory. You should instead create 
the label once and then update it

percentViewLab = Label(...)
percentViewLab.grid(...)

def viewPercent():
percentViewLab["text"] = percentScore

- You convert integers to integers.

>score = int(score + 1)

should better be written

 score = score + 1

or
 score += 1

- I rarely say that, but you have too many functions -- or rather you have 
written them in such a way that you need to call them in a special order. 
Personally I would omit the percentCheck function and the percentScore 
variable and rewrite viewPercent as

def viewPercent():
percentViewLab["text"] = score/total*100

or if you want to keep the function

def get_percent_score():
return score/total*100

def viewPercent():
percentViewLab["text"] = get_percent_score()

With both approaches viewPercent() can never show an out-of-date value.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Times Tables Program that constantly tells you that you are wrong!

[Peter Otten]

Matthew Polack wrote:

> *Question 2:*
> Is there a way to centre text within the text frame? I can change the font
> and size...but don't know how to centre it?

Ok, I wanted to know it myself, and found

https://stackoverflow.com/questions/42560585/how-do-i-center-text-in-the-tkinter-text-widget


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Times Tables Program that constantly tells you that you are wrong!

[Peter Otten]

Matthew Polack wrote:


> *Question 1:*
> 
> *Why cant the program check if 'answer' variable is correct?*

Change the line

result = "Sorry...you were wrong.

in your script to

result = "Sorry...you were wrong. Expected {!r}, but got 
{!r}".format(answer, response)

and find out yourself.

Hint: answer in the makeproblem() function is not a global variable.
 
> lib\tkinter\__init__.py", line 3269, in insert
> self.tk.call((self._w, 'insert', index, chars) + args)
> _tkinter.TclError: bad text index "1"

The first argument of the Text.insert() is a string in a special format. Read

http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/text-index.html
http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/text-methods.html

for the details. You have a mix of one float (which may work by accident) or 
two integers (which raises an exception).


> *Question 2:*
> Is there a way to centre text within the text frame? I can change the font
> and size...but don't know how to centre it?

I don't know, and I'm too lazy to google ;)

 
> txt = Text(root, width=35, height=10, wrap=WORD, font=("arial", 24,
> "bold")) txt.grid(row=12, columnspan=2, sticky=EW )
> 
> 
> *Question 3:*
> 
> My 'Score' check idea..may not actually work...haven't been able to get
> that far yet though!

The score variable is not declared global and not initialised. You cannot 
increment something that doesn't exist.

> Thanks so much for any clues.
> 
> - Matthew Polack
> 
> 
> 
> 
> 
> 
> 
> Matthew Polack | Teacher
> 
> 
> [image: Emailbanner3.png]
> 
> Trinity Drive  |  PO Box 822
> 
> Horsham Victoria 3402
> 
> p. 03 5382 2529   m. 0402456854
> 
> e. matthew.pol...@htlc.vic.edu.au
> 
> w. www.htlc.vic.edu.au
> 


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Query: lists

[Peter Otten]

Alan Gauld via Tutor wrote:

> On 14/08/18 09:11, Deepti K wrote:
>>  when I pass ['bbb', 'ccc', 'axx', 'xzz', 'xaa'] as words to the below
>> function, it picks up only 'xzz' and not 'xaa'
> 
> Correct because
> 
>> def front_x(words):
>>   # +++your code here+++
>>   a = []
>>   b = []
>>   for z in words:
>> if z.startswith('x'):
>>   words.remove(z)
> 
> You just changed the thing you are iterating over.
> By removing an elem,ent the list got shorter so the
> internal counter inside the for loop now points at
> the next item - ie it skipped one.
> 
> As a general rule never modify the thing you are
> iterating over with a for loop - use a copy or
> change to a while loop instead.
> 
> 
>>   b.append(z)
>>   print 'z is', z
>>   print 'original', sorted(words)
> 
> But it's not the original because you've removed
> some items.
> 
>>   print 'new', sorted(b)
>>   print sorted(b) + sorted(words)
> 
> But this should be the same as the original
> (albeit almost sorted).
> 
> PS. Since you only modify 'b' and 'words' you
> don't really need 'a'

For a simple solution you do need a and b: leave words unchanged, append 
words starting with "x" to a and words not starting with "x" to b.

Someone familiar with Python might do it with a sort key instead:

>>> sorted(['bbb', 'ccc', 'axx', 'xzz', 'xaa'],
... key=lambda s: not s.startswith("x"))
['xzz', 'xaa', 'bbb', 'ccc', 'axx']

If you want ['xaa', 'xzz', 'axx', 'bbb', 'ccc'] as the result
you can achieve that by sorting twice (Python's sorting is "stable") or by 
tweaking the key function.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Counting the # of iterations OR storing # values in a list

[Peter Otten]

Gautam Desai wrote:

> I am currently working with the lattice.py code attached. The code is a
> simulation that models the interactions between bacteria by producing a
> lattice output.
> 
> Once you run the code, there should be both blue and red bacteria that
> show
> on a lattice.  I can manipulate the size of the lattice to be an integer.
> In this case, I have set it to 250
> 
> However, there are different bacteria cells occupying each "pixel" on the
> lattice. I could count the number of red or blue bacteria on the output,
> but It would take an extremely long time

Counting doesn't take very long, at least for *only* 250**2 cells, and 
there's even a Counter in the stdlib. When I tried

$ python3 -i lattice.py 
>>> from collections import Counter
>>> Counter(np.ravel(my_lattice.lattice))
Counter({0.22951: 31306, 0.0025402: 31194})

there was no noticeable delay on my aging hardware. For really large arrays

>>> (my_lattice.lattice == 0.2295).sum()
31306

is probably faster.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Removing duplicates

[Peter Otten]

Oscar Benjamin wrote:

> On 1 August 2018 at 21:38, Roger Lea Scherer  wrote:
>>
>> I'm trying to get a list of tuples to be a float, a numerator, and a
>> denominator for all the fractions: halves, thirds, fourths etc up to
>> ninths. 1/2 returns the same float as 2/4, 3/6, 4/8. I would like to keep
>> only the 1/2. When I try (line 18) to "pop"  from the list I get a
>> "TypeError:
>> integer argument expected, got float". When I try (line 18)  to "remove"
>> from the list, nothing happens: nothing is removed and I do not receive
>> an error message.
>>
>> What do you think is a good way to solve this?
>>
>> Thank you as always.
>>
>> import math
>>
>> fractions = [(0, 0, 0)]
>>
>> for i in range(1, 10):
>> for j in range(1, 10):
>> if i < j:
>> x = i/j
>> if x not in fractions:
>> fractions.append((x, i, j))
>> sortedFrac =  sorted(fractions)
>>
>> print(sortedFrac)
>>
>> for i in range(len(sortedFrac)):
>> try:
>> if sortedFrac[i][0] == sortedFrac[i-1][0]: # so if the float
>> equals
>> the previous float
>> sortedFrac.pop(sortedFrac[i][0])   # remove the
>> second
>> float
>> else:
>> sortedFrac.append(sortedFrac[i][0])
>> except ValueError:
>> continue
> 
> Comparing floats for equality can be flakey. Sometimes two floats that
> should be equal will not compare equal e.g.:
> 
 0.01 + 0.1 - 0.1 == 0.01
> False

Do you know if there's a way to construct an example where

i/k != (n*i)/(n*k)

with preferrably small integers i, k, and n? Python's integer division 
algorithm defeats my naive attempts ;)

I had to resort to i/k == i/(k + 1)

>>> k = 10**18
>>> 1/k == 1/(k+1)
True

> This happens in this case because of intermediate rounding errors. I
> don't think that should affect you since you are doing precisely one
> floating point operation i/j to calculate each of your floats but
> unless you have a very solid understanding of binary floating point I
> would recommend that you shouldn't compare floating point values as in
> a==b.
> 
> For this particular problem I would use integer arithmetic or I would
> use the fractions module. Doing this with integers you should
> normalise the numerator and denominator by dividing out their GCD. The
> fractions module takes care of this for you internally.
> https://docs.python.org/3.7/library/fractions.html
> 
> Otherwise in general to remove duplicates you would be better of with
> a set rather than a list. If you only put x in the set and not i and j
> then the set will automatically take care of duplicates:
> https://docs.python.org/3.7/tutorial/datastructures.html#sets

And if you want to keep i and j you can use a dict:

fractions = {}
for i in range(1, 10):
for j in range(1, 10):
if i < j:
x = i/j
if x not in fractions:
fractions[x] = x, i, j
sorted_unique_fractions =  sorted(fractions.values())

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSL Error

[Peter Otten]

Saket Mehrotra wrote:

> Hi
> 
> I am trying to run import requests in a py file but I am getting below
> error  ssl.PROTOCOL_SSLv23: OpenSSL.SSL.SSLv23_METHOD,
> AttributeError: module 'ssl' has no attribute 'PROTOCOL_SSLv23'

If you start the Python interpreter and execute

>>> import ssl
>>> ssl.__file__
'/usr/lib/python3.4/ssl.py'

does it print something suspicious? You might be hiding the standard ssl.py 
with one you created yourself:

# touch ssl.py
$ python3
Python 3.4.3 (default, Nov 28 2017, 16:41:13) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.__file__
'/home/peter/mystuff/ssl.py'

In that case just rename the offending file (if you are using Python 2 
remember to remove the corresponding ssl.pyc).

>>>>
> 
> I tried to execute below from the terminal but it still remains
> unresolved.
> 
> sudo easy_install --upgrade pip
> 
> I've also had to run:
> 
> sudo pip uninstall pyopenssl
> 
> sudo pip install mozdownload
> 
> 
> Can you please help me .
> 
> Thanks & Regards
> Saket Mehrotra
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to add an Image in Grid mode with Python and Tkinter?

[Peter Otten]

Matthew Polack wrote:

> Hi,
> 
> Steadily trying to learn Python and Tkinter here with our students...
> 
> Am working on a simple 'Price of Cereal Crop' calculatorand have made
> a start with the code below...
> 
> I'm trying to simply add an image to our program to make the GUI more
> interesting...but most of the tutorials describe using the 'Pack'
> method not the grid method of layout...

Open the image with PIL, and then put the image into a Label or Button.
What layout manager you use for that widget afterwords doesn't really 
matter.
 
> and apparently you cannot use 'Pack' and 'Grid' in the one program...
> 
> Can anyone explain how I could add an image to this program? Or show me an
> example of a simple program that lets someone simply add an image and use
> the grid layout in combination.

Let's assume you want to show a picture stored in the file "wheat.jpg" on 
the button with the -- misleading! -- name wheatlabel. Then

> from tkinter import *

from PIL import Image, ImageTk

> root = Tk("Crop Calculator")

# The line above produced an exception over here; I think you wanted

root = Tk() 
root.title("Crop Calculator")

 
> # abutton = Button(root, text="Crop Prices in Australia")
> 
> #Main Heading Labels
> croplabel = Label(root, text="Crop Prices in Australia", fg="white", bg=
> "green")
> instruction = Label(root, text="Please select from the following crops",
> fg= "green", bg="white")
> blanklabel = Label(root, text="", fg="green", bg="white")
> blanklabel2 = Label(root, text="", fg="green", bg="white")
> statusbar = Label(root, text="Copyright 2018", fg="white", bg="black",
> relief='sunken')
> 
> #Answerlabel
> answerlabel=Label(root, text="wheatwords")
> 
> # abutton.pack(side='left', fill='both', expand=True)
> croplabel.grid(columnspan=12, sticky='ew')
> instruction.grid(row=1, columnspan=12, sticky='ew')
> blanklabel.grid(row=2, columnspan=12, sticky='ew')
> statusbar.grid(row=12, columnspan=12, sticky='ew')
> blanklabel2.grid(row=11, columnspan=12, sticky='ew')
> 
> 
> #List of Crops

wheat_image = Image.open("wheat.jpg")
wheat_photo = ImageTk.PhotoImage(wheat_image)

wheatlabel = Button(root, image=wheat_photo, fg="black", bg="yellow")

> peaslabel = Button(root, text="Peas", fg="white", bg="green")
> lupenslabel = Button(root, text="Lupens", fg="white", bg="brown")
> barleylabel = Button(root, text="Barley", fg="white", bg="orange")
> canolalabel = Button(root, text="Canola", fg="white", bg="red")
> sorghumlabel = Button(root, text="Sorghum", fg="white", bg="ivory3")
> 
> # Grid positioning of crops.
> wheatlabel.grid(row=4, column=1, sticky='ew')
> peaslabel.grid(row=4, column=7, sticky='ew')
> lupenslabel.grid(row=5, column=1, sticky='ew')
> barleylabel.grid(row=5, column=7, sticky='ew')
> canolalabel.grid(row=6, column=1, sticky='ew')
> sorghumlabel.grid(row=6, column=7, sticky='ew')
> 
> # Definitions
> 
> def wheatwords():
> print("Button one was pushed")
> 
> wheatlabel.configure(command=wheatwords)
> 
> 
> root.mainloop()
> 


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Dictionary viceversa

[Peter Otten]

Zachary Ware wrote:

> On Mon, Jul 30, 2018 at 12:20 PM Valerio Pachera  wrote:
>> I was looking to substiture the cicle for e in new_d like this:
>>   [ new_d[e].append(k) if e in new_d else new_d[e].append(k) for e in
>>   [ d[k] ]
>> but it can't work because 'new_d[e] = []' is missing.
> 
> Have a look at `dict.setdefault` and play around with it; I think it
> will help you do what you want.

Either that, or use a defaultdict:
 
>>> import collections
>>> room_access = collections.defaultdict(list)
>>> for user, rooms in users.items():
... for room in rooms:
... room_access[room].append(user)
... 
>>> room_access
defaultdict(, {'office-c': ['user3'], 'office-b': ['user2', 
'user1'], 'office-a': ['user3', 'user1']})


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Stuck on some basics re floats

[Peter Otten]

Matthew Polack wrote:

> Hi,
> 
> I'm a teacher trying to learn Python with my students.
> 
> I am trying to make a very simple 'unit calculator' program...but I get an
> error ..I think python is treating my num1 variable as a text string...not
> an integer.
> 
> How do I fix this?
> 
> Thanks!
> 
> - Matt
> 
> print ("How many inches would you like to convert? ")
> num1 = input('Enter inches here')

As you have guessed, at this point num1 is a string. You can convert it to a 
float with

num1 = float(num1)

> print ("You have entered",num1, "inches")
> convert = num1 * 2.54
> print ("This is", convert, "centimetres")
> 
> 
> Traceback (most recent call last):
>   File "convert.py", line 10, in 
> convert = num1 * 2.54
> TypeError: can't multiply sequence by non-int of type 'float'

Note that int is mentioned here because Python can multiply int with str, 
though the result might surprise you:

>>> "spam" * 3
'spamspamspam'

So * also works as the repetition-operator just like + is also used to 
concat strings.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CSV Read

[Peter Otten]

Giulia Marcoux wrote:

> Hello,
> 
> I am a student learning python, and i am running into some difficulties. I
> am tryign to read a csv file that has different delimiters for different
> rows: Example:
> 
> Format:V1.1
> Model: R
> Step Size: 10mm
> Distance: 10cm
> Gain: 1000
> 
> X,Y
> 1,3
> 2,5
> 6,5
> 5,7
> 
> For the X,Y data, I have no problem getting it into the right cells,
> however I cannot separate the values after the colon for the first few
> rows. Its either one or the other. I am using pandas to read my CSV file.
> So far I have:
> 
> d=pd.read_csv('read.csv',delimiter=',',skiprows=0)
> 
> When I replace the delimiter for ':', it no longer works, and if i put
> sep=',|:' it does not work either. In the end I need to plot the x,y
> values and i will need some of the info in the rows above like the gain to
> manipulate the y data. If you have any tips for me I would be very
> grateful!

Rather than forcing both blocks into the same data structure I would read 
the header with the "key: value" lines into a separate object.

If the header and the X,Y pairs are separated by an empty line read until 
that and then feed the file object to pandas.read_csv() to read those pairs:

import itertools
import pandas as pd


def header(f):
"""Pass on lines from f; stop at the first whitespace-only line.
"""
return itertools.takewhile(str.strip, f)


def pairs(lines, sep=":"):
"""Split lines into (key, value) pairs.

Remove leading/trailing whitespace from value.
"""
for line in lines:
key, value = line.split(sep, 1)
yield key, value.strip()


with open("data.txt") as f:
metadata = dict(pairs(header(f)))
df = pd.read_csv(f)

print(metadata)
print(df)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] adding numpy to pandas

[Peter Otten]

Mats Wichmann wrote:

> On 06/20/2018 02:04 PM, Glenn Schultz wrote:
>> All,
>> 
>> I have a pandas dataframe and a predict result (numpy array) of a
>> classifier [[0,1],[1,0]].  What I would like to do is as the positive to
>> the pandas dataframe.  I use predict[:,1] to slice the postive from
>> numpy which gives me a row of the result.  but I cannot concat to the
>> pandas df['result'] = predict[:,1] does not work and I have tried
>> various ways to do this with no result.  I am missing something here.
> 
> You should take a look here:
> 
> https://pandas.pydata.org/community.html
> 
> History has indicated that the Python tutor group isn't overloaded with
> Pandas experts. You may still get an answer here, but that page suggests
> the preferred places from the community to interact with to get good
> answers.  There's also a Google Groups which doesn't seem to be
> mentioned on the page:
> 
> https://groups.google.com/forum/#!forum/pydata

Regardless of the chosen forum, try to be as precise as possible with your 
problem description. It really can't get any worse than "does not work".

I tried but failed to reproduce your problem from what little information 
you provide:

>>> a = np.array([[0,1],[1,0]])
>>> df = pd.DataFrame([[1,2], [3,4]], columns=["a", "b"])
>>> df["result"] = a[:,1]
>>> df
   a  b  result
0  1  2   1
1  3  4   0

Please take the time to read http://sscce.org/ to learn how you can improve 
your question. You'll be rewarded with better answers from us or by the real 
experts elsewhere.

Thank you.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor