Re: [Tutor] python question

[Steven D'Aprano]

On Sun, Aug 18, 2019 at 12:35:52PM +0800, Thejal Ramesh wrote:
> Hi, i have a question regarding this question. I'm not quite sure what the
> question is asking.

Ask your tutor. We can help you with learning Python the programming 
language, not graph theory.

https://en.wikipedia.org/wiki/Graph_theory


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class functions/staticmethod?

[Steven D'Aprano]

On Wed, Aug 14, 2019 at 09:58:35AM +1000, Cameron Simpson wrote:
> On 11Aug2019 22:58, James Hartley  wrote:
> >I am lacking in understanding of the @staticmethod property.
> >Explanation(s)/links might be helpful.  I have not found the descriptions
> >found in the Internet wild to be particularly instructive.
> 
> You have received some answers; to me they seem detailed enough to be 
> confusing.

Its only confusing if you don't work your way through it carefully and 
systematically. There's a lot to understand, but if you don't understand 
it, Python's behaviour in this case seems counter-intuitive and hard to 
follow.

Python makes the behaviour of regular instance methods so simple and 
intuitive, it can be quite a blow when you try to do something that 
isn't.


> I think of things this way: what context does a method require?  Not 
> everything needs the calling instance.
> 
> Here endeth the lesson.

Given that you go on to write almost another 150 lines of explanation, I 
think a better description would be "Here *begins* the lesson" *wink*


Your lesson, I think, assumes that it is obvious that staticmethods 
don't have access to the calling instance, or its class. But if you look 
at James' code, I think you will agree that he's assuming that 
staticmethods *do* have access to the calling class, and is perplexed by 
the fact that the look-up of class variables (class attributes) fails.

If James comes from a Java background, he's probably assuming that 
static methods do have access to the class variables, using undotted 
names:

class K(object):
attr = 1
@staticmethod
def foo():
return attr

In Java, K.foo() would return 1.

Your lesson gives us no clue why James' first method, "dimensions()", 
which he describes as a "class method", isn't a class method and doesn't 
actually work correctly, even though it appears to at first glance.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class functions/staticmethod?

[Steven D'Aprano]

Part 3.


On Sun, Aug 11, 2019 at 10:58:37PM -0500, James Hartley wrote:


> from collections import namedtuple
> 
> class Foo():
> Dimensions = namedtuple('Dimensions', ['height', 'width'])
> _dimensions = Dimensions(3, 4)
> 
> def dimensions():
> print('id = {}'.format(id(Foo._dimensions)))
> return Foo._dimensions
> 
> @staticmethod
> def dimensions1():
> print('id = {}'.format(id(_dimensions)))
> return _dimensions


In part 2, I explained that we can re-write the dimensions() method to 
work correctly using the @classmethod decorator:

@classmethod
def dimensions(cls):
print('id = {}'.format(id(cls._dimensions)))
return cls._dimensions


Another benefit of doing this is that it will now work correctly in 
subclasses.

class Bar(Foo):  # inherit from Foo
_dimensions = (3, 4, 5, 6)  # Override the parent's "dimensions".


Using your definition, Bar.dimensions() will return Foo._dimensions 
instead of Bar._dimensions. But using the classmethod version works as 
expected.

So why doesn't the staticmethod version work correctly? Its all to do 
with the way variable names are resolved by the interpreter.

If you are used to Java, for example, you might expect that "class 
variables" (what Python calls "class attributes") are part of the scope 
for methods:


spam = 999  # Global variable spam.

class MyClass(object):
 spam = 1  # Class attribute ("variable") spam.

 def method(self):
 return spam

instance = MyClass()


If you are used to Java's rules, you would expect that instance.method() 
will return 1, but in Python it returns the global spam, 999.

To simplify a little, the scoping rules for Python are described by the 
LEGB rule:

- Local variables have highest priority;
- followed by variables in the Enclosing function scope (if any);
- followed by Global variables;
- and lastly Builtins (like `len()`, `zip()`, etc).

Notice that the surrounding class isn't included.[1] To access either 
instance attributes or class attributes, you have to explicitly say so:

 def method(self):
 return self.spam

This is deliberate, and a FAQ:

https://docs.python.org/3/faq/design.html#why-must-self-be-used-explicitly-in-method-definitions-and-calls


Using Java's scoping rules, the staticmethod would have worked:

@staticmethod
def dimensions1():
print('id = {}'.format(id(_dimensions)))
return _dimensions

because it would see the _dimensions variable in the class scope. But 
Python doesn't work that way. You would have to grab hold of the class 
from the global scope, then grab dimensions:

@staticmethod
def dimensions1():
_dimensions = Foo._dimensions
print('id = {}'.format(id(_dimensions)))
return _dimensions


If you are coming from a Java background, you may have been fooled by an 
unfortunate clash in terminology. A "static method" in Java is closer to 
a *classmethod* in Python, not a staticmethod.

The main difference being that in Java, class variables (attributes) 
are automatically in scope; in Python you have to access them through 
the "cls" parameter.



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Create Logging module

[Steven D'Aprano]

On Thu, Aug 01, 2019 at 05:11:04PM +0800, Sinardy Xing wrote:

> I have error look like in the wrapper.
> 
> Can someone point to me where is the issue

No, but you can.

When the error occurs, Python will print a traceback containing a list 
of the lines of code being executed, and the final error. You should 
read that error, especially the last line, and it will tell you the line 
of code that failed and give you a clue why it failed.

We can help you if you copy and paste the full traceback, starting with 
the line "Traceback..." to the end.

Don't take a screen shot or photo and send that, instead copy and paste 
the text into your email.

(P.S. please reply to the mailing list, not to me personally.)


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fw: CSC1010H 4th assignment

[Steven D'Aprano]

On Thu, Jul 25, 2019 at 07:04:52PM +, Mahima Daya wrote:

> hi there, please could you assist.

Yes, certainly. If you have questions about Python, you can ask 
concrete, specific questions. If you ask vague questions like "please 
help" expect to be ignored or told to do your own homework.

You should start by learning how to ask good questions about your code. 
Please read this:

http://sscce.org/

It is written mostly for Java programmers but it applies equally to any 
programming language, including Python.

And remember that asking people to work on your assignments without 
crediting them is almost certainly a violation of your university's 
standards of academic ethics.

-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A question about daunting KeyError

[Steven D'Aprano]

Hi Taishi, and welcome.

On Wed, Jul 24, 2019 at 03:21:03AM +, TAISHI KAWAMURA wrote:

[...]
> I'm suspecting that Anaconda might keep raising the error, or simply 
> there are bugs in the codes. However, I can't be sure what the real 
> cause is.

Almost certainly it will be a bug in your code, or unexpected values in 
your data. It is possible that it is a bug in Anaconda, but not very 
likely.

Start by removing Anaconda from the problem: what happens if you run 
your code using the plain Python interpreter, without Anaconda involved? 
>From the operating system command line, run something like this:

# Windows CMD.EXE
py path/to/myscript.py

# Linux or Mac OS shell
python path/to/myscript.py


Of course the example command you run will depend on the version of 
Python you are using, and the name and path to your script.

If the error goes away without Anaconda involved, then it is *possible* 
that it is a bug in Anaconda. But I expect the error will remain.

Then show us the full traceback of the error. Copy and paste the FULL 
exception message, starting with the line "Traceback" and ending with 
the KeyError and error message, something like this but probably longer:

Traceback (most recent call last):
  File "", line 1, in 
KeyError: (1, 2, 3)


Once you know the line where the error is occurring, you could introduce 
some debugging code:

try:
result = data[key]
except KeyError:
log(data)
log(key)
raise


or use the debugger to investigate why the key is missing.

You should also read this:

http://sscce.org/


It is written for Java programmers, but it applies to Python too.




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Generator expressions

[Steven D'Aprano]

On Wed, Jul 24, 2019 at 11:26:26AM +1200, David L Neil wrote:

> Clarifying the difference/similarity in appearance between a generator 
> expression and a tuple, it might help to think that it is the comma(s) 
> which make it a tuple!

David makes an excellent point here. Except for the special case of the 
empty tuple, it is *commas* that create tuples, not parentheses. The 
round brackets are used for grouping, and are optional:

py> a = 1, 2, 3
py> type(a)



The only times you *need* parens around a non-empty tuple is to group 
the items or to avoid ambiguity, e.g. a tuple inside a tuple:

a = 1, 2, (3, 4, 5), 6
assert len(a) == 4

or when passing a literal tuple as argument to a function:

function(1, 2, 3, 4, 5, 6)# six arguments
function(1, 2, (3, 4, 5), 6)  # four arguments




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Generator expressions

[Steven D'Aprano]

On Tue, Jul 23, 2019 at 10:36:01PM +0530, Animesh Bhadra wrote:
> Hi All,
> 
> Need one help in understanding generator expression/comprehensions.
> 
> This is my sample code.

Lots of missing spaces in your code! Don't forget to hit the space bar 
between words :-)

Also missing indentation, which is probably Gmail being "helpful".

I'm just going to fix the obvious typos without comment.


> # This code creates a generator and not a tuple comprehensions.
> my_square =(num*num for num in range(11))
> print(my_square) #  at 0x7f3c838c0ca8>
> # We can iterate over the square generator like this.
> try:
> while True:
> print(next(my_square)) # Prints the value 0,1,4
> except StopIteration as SI:
> print("Stop Iteration")
> # Another iteration
> for x in my_square:
> print(x) # This prints nothing.


> Does the generator exhausts its values when we run the iterator once?

Correct.


> Lastly any specific reason for not having a tuple comprehensions?

(1) History and (2) because tuple comprehensions are not useful enough 
to dedicate syntax to them.


Originally, Python introduced *list comprehensions* using square bracket 
syntax:

[expression for x in values]

which created a list. If I remember correctly, I think this was about 
Python 2.2. 

Tuples are mostly intended for *small* numbers of heterogeneous values, 
like a struct or record in other languages:

# Typical example of tuple
(1, "Hello", 4.5)

while lists are intended for potentially large numbers of homogeneous 
values, like an array of data:

[2.3, 4.5, 0.9, 7.2, 9.3, 4.5, 6.1, 0.2, 8.7, ... # and many more

(heterogeneous = "different types", homogeneous = "same types")

That makes lists a really good fit for comprehensions, and tuples a bad 
fit. So it was decided that the need for a direct tuple comprehension 
was pretty low. If you wanted a tuple, you could easily get one:

tuple([expression for x in values])

and that was good enough without wasting useful syntax on something that 
would hardly ever be needed.

It would be a tiny bit more expensive to run (first you create a list, 
then copy it to a tuple, and then garbage collect the list) but so what? 
Not every rare operation needs to be optimized.

A few years later, generator comprehensions were added as a lazy version 
of list comprehensions. List comprehensions produce all the values up 
front, while generator comprehensions produce them one at a time as 
needed. The syntax chosen used round brackets ( ... ) instead of square 
brackets [ ... ] but otherwise was identical.

What other syntax could have been chosen? Curly brackets are used for 
dicts (Python 2 and 3) and sets (Python 3 only), so they are unsuitable, 
and later on in Python 3 we gained dict and set comprehensions as well.

Had we wasted the ( ... ) syntax on tuples, which hardly anyone would 
have ever used, we would have missed out on the super-useful lazy 
generator comprehensions.


 
> Have checked this link, but could not understood the reason?
> 
>  * 
>  
> https://stackoverflow.com/questions/16940293/why-is-there-no-tuple-comprehension-in-python

Don't believe everything you read on Stackoverflow. There's lots of 
utter nonsense on that page, for example:

"The answer is obviously because tuple syntax and parenthesis are 
ambiguous"

is rubbish. The parser can tell the difference between tuples and 
parentheses. Tuples use *commas*, not parentheses, except for the 
empty tuple () which is a special case. There's no ambiguity in Python.

The top rated answer

"parentheses were already taken for … generator expressions"

is incorrect, because generator expressions came second, not first. 
Python could have added tuple comprehensions at the same time it added 
list comprehensions, but the developers decided not to:

- it wasn't clear how useful comprehensions would be; they were really 
useful in Haskell, but people weren't sure if Python developers would 
take to them (they sure did!);

- list comprehensions are clearly useful, but tuple comprehensions not 
so much. Not ever single use has to get its own special syntax.

Of course, *later on* once Python had generator comprehensions using 
parentheses, we couldn't easily add tuple comprehensions because 
there's no good syntax left. But why would we want them?



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to convert string to date time format?

[Steven D'Aprano]

On Fri, Jul 19, 2019 at 10:44:36PM -0400, C W wrote:
> Hello all,
> 
> I have a date time string that looks like the following.
> 
> 02015-07-01 00:01:44.538420-08:00
> 12015-07-01 00:27:58.717530-08:00
> 22017-07-01 07:07:48.391376-08:00

I assume that the leading number and spaces "0" etc are NOT part of 
the strings.

 
> I have tried the following two different methods, both did not work.
> Method one: pandas
> import pandas as pd
> stamp = pd.to_datetime(my_string, format='%Y%m%d %H:%M:%S')
> 
> Method two: datetime package
> from datetime import datetime
> datetime.strptime(my_string, '%Y-%m-%d %H:%M:%S')
> 
> 
> Are both ways suppose to work?

Not unless the string format matches the actual string. You can't expect 
to convert a string unless it matches the format.


> Also, does it matter if there are decimals
> after seconds?

Of course it matters.

Did you read the error message? The single most important skill for a 
programmer is to READ THE ERROR MESSAGE and pay attention to what it 
tells you went wrong:

py> datetime.strptime('2015-07-01 00:01:44.538420-08:00', '%Y-%m-%d %H:%M:%S')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.5/_strptime.py", line 346, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: .538420-08:00


See how the error message tells you that it couldn't convert the string 
because there is data left over at the end. The first thing to do is 
handle the microseconds.

Googling gets the answer: use "%f" as the code for fractional seconds.

https://duckduckgo.com/?q=strptime+seconds+with+decimals


py> datetime.strptime('2015-07-01 00:01:44.538420-08:00', '%Y-%m-%d 
%H:%M:%S.%f')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.5/_strptime.py", line 346, in _strptime
data_string[found.end():])
ValueError: unconverted data remains: -08:00


Now we're making progress! The error message has changed. Now you just 
need to decide what the "-08:00" part means, and change the format 
string appropriately.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pass arg to list

[Steven D'Aprano]

On Thu, Jul 18, 2019 at 11:34:09AM -0700, Anirudh Tamsekar wrote:

> My script below is blowing index out of range after adding the args.

> version = sys.argv[1]

> Traceback (most recent call last):
>   File "/Users/atamsekar/Projects/PulseSO/trunk/swrelease/samplexls.py",
> line 5, in 
> version = sys.argv[1]
> IndexError: list index out of range


The obvious question is, are you passing command line arguments to your 
script?

How are you calling your script? If you are running it from an IDE you 
need to tell the IDE to include command line arguments.

If you run it like this, from the operating system's command line:

python samplexls.py

then there are no command line arguments and sys.argv[1] will be out of 
range. You need to run something like this:

python samplexls.py argument1 argument2


Try putting:

print(sys.argv)

at the start of your script and see what is there.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] raising exceptions in constructor code?

[Steven D'Aprano]

On Tue, Jul 16, 2019 at 04:29:15PM -0500, James Hartley wrote:
> I ask this having more C++ knowledge than sense.
> 
> There is an adage in the halls of everything Stroustrup that one needs to
> think about how resource allocation will be unwound if an exception is
> thrown.  This gets watered down to the mantra "Don't throw exceptions from
> within constructors."  Does this carry over to Python?  I'm trying to
> develop a Pythonistic mindset as opposed to carrying over old baggage...

No, it is perfectly safe to raise exceptions from within the Python 
constructors, whether you are using __new__ (the true constructor) or 
__init__ (the initialiser).

The only tricky part is if you allocate resources external to the 
object, like this:


class Weird(object):
openfiles = []
def __new__(cls, fname):
f = open(fname)
cls.openfiles.append(f)
# New instance:
instance = super().__new__(cls)
if condition:
raise ValueError
return instance


Even if the __new__ constructor fails, I've kept a reference to an open 
file in the class. (I could have used a global variable instead.) That 
would be bad. But notice I had to work hard to make this failure mode, 
and write the code in a weird way. The more natural way to write that^1 
would be:

class Natural(object):
def __init__(self, fname):
self.openfile = open(fname)
if condition:
raise ValueError

Now if there is an exception, the garbage collector will collect the 
instance and close the open file as part of the collection process. 
That might not be immediately, for example Jython might not close the 
file until interpreter shutdown. But the earlier example will definitely 
leak an open file, regardless of which Python interpreter you use, while 
the second will only leak if the garbage collector fails to close open 
files.


Here's a better example that doesn't depend on the quirks of the garbage 
collector:

class Leaky(object):
instances = []
def __init__(self):
self.instance.append(self)
if random.random() < 0.1:
raise ValueError

This will hold onto a reference to the instance even if the initialiser 
(constructor) fails. But you normally wouldn't do that.

class NotLeaky(object):
def __init__(self):
if random.random() < 0.1:
raise ValueError


try:
x = NotLeaky()
except ValueError:
pass


Now either the call to NotLeaky succeeds, and x is bound to the 
instance, or it fails, and x is *not* bound to the instance. With no 
references to the newly-created instance, it will be garbage collected.




^1 Actually that's not too natural either. It is not usually a good idea 
to hold onto an open file when you aren't actively using it, as the 
number of open files is severely constrained on most systems. 



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Object references in Python

[Steven D'Aprano]

On Tue, Jul 16, 2019 at 12:08:10PM +, AHIA Samuel wrote:

> Please what are references in Python development?

x = 19
y = x

The name "x" is a reference to the int 19; the name y is a reference to 
the same int.

x = "Hello world"

Now the name "x" is a reference to the string "Hello world". y remains a 
reference to the int 19.

x = None

Now the name "x" is a reference to the None object.

x = [100, 200, 300]

Now the name "x" is a reference to a list with three items:

- The first item of x, x[0], is a reference to the int 100.
- The second item of x, x[1], is a reference to the int 200.
- The third item of x, x[2], is a reference to the int 300.

y = x

Now y is no longer a reference to the int 19 as before, but is a 
reference to the same list that x is a reference to. There are now two 
references to the list object: x and y.

(If you are a C programmer, you can think of x and y both being pointers 
to the same list. This is not completely correct, but it isn't far 
wrong.)

Since x and y are references to the same list, we can equally say:

- The first item of y, y[0], is a reference to the int 100.
- The second item of y, y[1], is a reference to the int 200.
- The third item of y, y[2], is a reference to the int 300.

x.append(None)

Now the name x is still a reference to the same list as before, except 
that we have added a new item to the end of the list:

- The fourth item of x, x[3], is a reference to the None object.
- The fourth item of y, y[3], is a reference to the None object.

Since both x and y are references to the same list, any change to the x 
list is a change to the y list (since they are the same).


class Parrot:
def __init__(self, colour="blue"):
self.colour = colour
def speak(self):
 print("Polly wants a cracker!")


Now the name "Parrot" is a reference to the class "Parrot".

x = Parrot()

Now x is a reference to a Parrot instance. y remains a reference to the 
list.

x.colour is a reference to the string "blue" (by default).

x.speak is a reference to the "speak" method of Parrot objects.



Does this help?



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Multiprocessing with many input input parameters

[Steven D'Aprano]

On Fri, Jul 12, 2019 at 11:53:16AM +, Shall, Sydney via Tutor wrote:
> Thanks Mike,
> 
> But I am still not clear.

Neither is your question.

> do I write:
> 
> def f([x,y,z]) ?
> How exactly do one write the function and how does one ensure that each 
> positional argument is accounted for.

Is your function taking three seperate arguments, or a single argument 
that must be a list of exactly three items?

(1) Three seperate arguments:

def function(a, b, c):
# Write the body of the function.
...


That's all you need do, as the interpreter will ensure it is only called 
with three arguments.


(2) One list argument with three items:

def function(alist):
if isinstance(alist, list):
if len(alist) < 3:
raise ValueError("too few arguments in alist")
elif len(alist) > 3:
raise ValueError("too many arguments in alist")
else:
a, b, c = alist  # Unpack the three items from the list.
# Write the body of the function here.
...
else:
raise TypeError('expected a list')



Python will ensure your function is only called with a single argument. 
The rest is up to you.




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Multiprocessing with many input input parameters

[Steven D'Aprano]

On Sat, Jul 13, 2019 at 09:59:16AM +1000, Cameron Simpson wrote:

> Mike has probably confused this with tuples. Because tuples are 
> delineated with parentheses, there is ambiguity between a tuple's 
> parentheses and normal "group these terms together" parentheses.

There are no "tuple parentheses". Tuples are created by the *comma*, 
not the parens. The only exception is the empty tuple, since you can't 
have a comma on its own.

x = ()# Zero item tuple.
x = 1,# Single item tuple.
x = 1, 2  # Two item tuple.


Any time you have a tuple, you only need to put parens around it to 
dismbiguate it from the surrounding syntax:

x = 1, 2, (3, 4, 5), 6 # Tuple containing a tuple.
function(0, 1, (2, 3), 4)  # Tuple as argument to a function.


or just to make it more clear to the human reader.


> Here is a 2 element tuple:
> 
>  (9, 7)
> 
> How does one write a one element tuple? Like this:
> 
>  (9,)

To be clear, in both cases you could drop the parentheses and still get 
a tuple:

9, 7

9,

provided that wasn't in a context where the comma was interpreted as 
something with higher syntactic precedence, such as a function call:

func(9, 7)# Two integer arguments, not one tuple argument.
func((9, 7))  # One tuple argument.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] collections and mappings

[Steven D'Aprano]

On Fri, Jun 21, 2019 at 10:01:43AM +1000, mhysnm1...@gmail.com wrote:

> I have reviewed the collection module and do not understand mappings. I have
> seen this in other languages and have never got the concept. Can someone
> explain this at a very high level.  

It might help if you remember that dicts are a kind of mapping.

Another name for dicts are "associative arrays".

In the most general terms, a mapping is an association between one or 
more pieces of information and another one or more pieces of 
information. That is so general as to be completely abstract, and I 
think it will be more useful to give some concrete examples.

If you are American, you probably have a social security number. There 
is a mapping between your social security number and you, the person: 
the government associates data about you with the social security 
number.

If you have used a regular paper dictionary, it is a mapping between 
words and definitions. We say the word maps to the definition.

In Python terms, we would use a dict, using the word as the key and the 
definition as the value. For example:

{"Childhood": 
The period of human life intermediate between the idiocy 
of infancy and the folly of youth—two removes from the sin of 
manhood and three from the remorse of age.""",

 "Debt":
"""An ingenious substitute for the chain and whip of the 
slave-driver.""",

 "Piano":
"""A parlor utensil for subduing the impenitent visitor. It 
is operated by depressing the keys of the machine and the 
spirits of the audience.""",

 "Quotation":
"""The act of repeating erroneously the words of another. 
The words erroneously repeated."""
}




Does this help?




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] difference between array([1,0,1]) and array([[1,0,1]])

[Steven D'Aprano]

On Thu, Jun 20, 2019 at 08:39:35PM -0300, Markos wrote:
> Hi,
> 
> I'm studying Numpy and I don't understand the difference between
> 
> >>>vector_1 = np.array( [ 1,0,1 ] )
> 
> with 1 bracket and
> 
> >>>vector_2 = np.array( [ [ 1,0,1 ] ] )
> 
> with 2 brackets

I'm not really sure what you don't get here. The first is a one 
dimensional vector with three items, the second is a two dimensional 
array with one row and three columns. But you know that, because you 
checked the shape of each one:

> >>>vector_1.shape
> (3,)

> >>>vector_2.shape
> (1, 3)

If you want a 2D shape, you need to provide a 2D argument. If you want a 
3D shape, you need a 3D argument. And so forth.


> I thought that both vectors would be treated as an matrix of 1 row and 3 
> columns.

Why 1x3 rather than 3x1?


> Why this difference?

Because that's how numpy arrays are designed to work. I'm not sure what 
sort of answer you expect.



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] word printing issue

[Steven D'Aprano]

Hi Sean,

Your subject line says "word printing issue", but the body of your email 
says nothing about an issue printing words.

Have you tried print(word)?

Further comments and questions below.


On Thu, Jun 20, 2019 at 08:44:06PM +1000, mhysnm1...@gmail.com wrote:
> I have a list of strings that I want to break them into separate words, and
> a combination of words then store them into a list. Example below of a
> string:
> 
> "Hello Python team".
> 
> The data structure:
> 
> [ ['Hello'],
>   ['Hello', 'Python'],
>   ['Hello', 'Python', 'team'],
>   ['Python'],
>   ['Python', 'team'],
>   ['team'] ]


I've taken the liberty of correcting some obviouis typos in the data 
structure.


Sean wrote:

> I want to know if there is a better method in doing this without the
> requirement of a module.

Better than what? How are you doing it now?

Why don't you want to use a module?


Sean wrote:

> Eventually I want to count the number of hits in a
> list of strings based upon the word combinations regardless where they are
> located in the string. This last part is something I am struggling with to
> get results that are correct.

I'm struggling to understand what you mean. I don't understand what the 
word combinations part has to do with the problem.

If all you want to do is count each word, you could try this:

from collections import Counter
text = "Hello Python team"
c = Counter(text.split())
print(c)


which will print something like:

Counter({'team': 1, 'Python': 1, 'Hello': 1})

(the order of the counts may be different).


> I have stored them in a dictionary and get
> unexpected totals. 

Either your expectations are wrong, and the totals are correct, or your 
expectations are correct, and the code counting them is wrong.

Without knowing either your expectations or the code counting the 
totals, I cannot guess which is the case.


> Was thinking of using collection and still working on it.
> If the above could be improved. I would be grateful.

Without knowing what you are doing, it is hard to suggest improvements.

"Dear cooking experts, I'm baking a cake and it turned out all wrong. 
What can I do to make it better? Thanks in advance."



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Differences between while and for

[Steven D'Aprano]

On Sat, Jun 15, 2019 at 02:53:43PM +1000, mhysnm1...@gmail.com wrote:
> All,
> 
>  
> 
> In C, Perl and other languages. While only uses a conditional statement and
> for uses an iteration. In python while and for seems to be the same and I
> cannot see the difference.

Python ``while`` uses a conditional statement, and Python ``for`` uses 
iteration. Python's ``for`` is like "foreach" in some other languages.

while condition: ...

for x in values: ...


> Python does not have an until (do while) where
> the test is done at the end of the loop. Permitting a once through the loop
> block. Am I correct or is there a difference and if so what is it?

Correct, there is no "do until" in Python.

> Why doesn't Python have an until statement?

Because Guido didn't want one :-)

Because it is unnecessary: any "do until" can be written as a regular 
while loop, using a break:

# do...until with test at the end
while True:
do_something()
if test:
   break


# "loop and a half"
# https://users.cs.duke.edu/~ola/patterns/plopd/loops.html#loop-and-a-half
while True:
do_something()
if test:
break
do_something_else()



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Download audios & videos using web scraping from news website or facebook

[Steven D'Aprano]

On Fri, Jun 14, 2019 at 11:35:53AM +0500, Sijin John wrote:

> I am trying to Download audios & videos using web scraping from news 
> website (eg: https://www.bbc.com/news/video_and_audio/headlines) or 
> Facebook & I could't. So in real scenario is it really possible to 
> download audios/videos using python code ?

Please don't mistake "I don't know how to do this" for "this cannot be 
done".

https://youtube-dl.org/

Scraping websites, especially scraping them for videos, can be *very* 
complex.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] would someone please explain this concept to me

[Steven D'Aprano]

In case you are still confused, we can simplify the whole process by 
eliminating the unnecessary use of an external module.

blank = {}
feeds = {}

feeds['a'] = blank
feeds['a'][1] = "Hello"

print(blank)
# will print {1: 'Hello'}

We can see that the dict named "blank" and the dict named "feeds['a']" 
are actually the same dict:

print(feeds['a'] is blank)  # Not the same as == (equals).

In case you still need more evidence:

feeds['b'] = blank
feeds['b'][1] = "Goodbye"
print(blank)
print(feeds)

will print:

{1: 'Goodbye'}
{'a': {1: 'Goodbye'}, 'b': {1: 'Goodbye'}}


(possibly the a and the b will be swapped around).



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] would someone please explain this concept to me

[Steven D'Aprano]

On Tue, Jun 04, 2019 at 11:37:23PM +, nathan tech wrote:

> globals.py:
> 
> feeds={}
> blank_feed={}
> blank_feed["checked"]=1
> blank_feed["feed"]=0

That is more easily, and better, written as:

feeds = {}
blank_feed = {"checked": 1, "feed": 0}


> main file:
> 
> import globals as g
> # some code that loads a feed into the variable knm

Do you mean something like this? If so, you should say so.

knm = "some feed"


> g.feeds[link]=g.blank_feed;

What's "link" here? And there is no need for the semi-colon.

> g.feeds[link]["feed"]=knm

Right... what the above line does is *precisely* the same as 

g.blank_feed["feed"] = knm

Follow the program logic. I'm inserting 1970s BASIC style line numbers 
to make it easier to discuss the code, remember that you can't actually 
do that in Python.

10: g.feeds[link] = g.blank_feed
20: g.feeds[link]["feed"] = knm

Line 10 sets g.feeds[link] to the dict "blank_feed". *Not* a copy: you 
now have two ways of referring to the same dict:

"g.blank_feed" and "g.feeds[link]"

both refer to the one dict, just as "Nathan" and "Mr Tech" are two ways 
of referring to the same person (you).

So line 20 does this:

- look for the name "g", which gives the "globals.py" module;

- inside that module, look for the name "feeds", which gives
  the "feeds" dict;

- look inside that dict for the key "link" (whatever value
  that currently holds), which by line 10 has been set to
  the same dict "blank_feed".

- inside the blank_feed dict, set key "feed" to "knm".



> #in the below code, the variable link has a different value:
> # load a feed into the variable r

Something like this?

r = "a different feed"

> g.feeds[link]=g.blank_feed

Now you have *three* ways of naming the same dict:

"g.blank_feed", "g.feeds[link]", "g.feeds[different_link]"

but they all point to the same dict.

> g.feeds[link]["feed"]=r
> 
> 
> Now at this point, python would set the first loaded feed to the same 
> thing as the second loaded feed. It also set g.blank_feed to the second 
> feed, as well.

No, there is only one feed in total. You just keep updating the same 
feed under different names.


> I replaced the last three lines with this:
> 
> # load a feed into the variable r
> g.feeds[link]=g.blank_feed;
> g.feeds[link]["feed"]=r
> 
> And it works.

I don't see any difference between the replacement code and the original 
code. The code you show does exactly the same thing.


> but why does it work?
> 
> Why does that semi unlink all the variables?

Semi-colon?

It doesn't. You must have made other changes as well, semi-colons don't 
have any runtime effect. They are *purely* syntax to tell the parser to 
seperate multiple statements on one line.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interactive editing of variables.

[Steven D'Aprano]

Hi Sean,

On Sat, Jun 01, 2019 at 12:53:00PM +1000, mhysnm1...@gmail.com wrote:
> I have had a look and cannot find an example where I can interactively edit
> a content of a variable at the command line. I do not want to use GUI at
> all. As this is a simple program only requiring CLI. I have no problems
> showing the prompt, but cannot insert text into the edit (input) area. Any
> ideas?

Let me see if I understand what you are asking for.

The built-in function "input()" takes one argument, the prompt, and lets 
the user type a response. So if I run the function:

input("What is your name?")

the terminal will display the prompt, and put the text cursor next to 
it. That is how things work now.

I think what you are asking for is the ability to pass a second argument 
to the function, like this:

input("What is your name?", "George")

and the terminal will display the prompt, then place the word "George" 
next to the prompt in the edit-zone. The user (you) can then backspace 
over the word and type something new, or otherwise edit it.

Is that what you want?

If so, then on Linux or Unix systems, you can do this:


import readline

def myinput(prompt='', initial=''):
readline.set_startup_hook(lambda: readline.insert_text(initial))
try:
response = input(prompt)
finally:
readline.set_startup_hook(None)
return response


The bad news is that this only works on Linux and other Unix systems 
with readline installed (which nearly all of them do). It won't work 
under Windows.

But if you install the third-party library pyreadline, you *may* be able 
to use that instead:

import pyreadline as readline

and the rest might work. (I don't have Windows and can't try it.)


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Case Insensitive Globing

[Steven D'Aprano]

On Sun, May 19, 2019 at 10:37:56AM +1000, Steven D'Aprano wrote:

> That's not quite right -- case sensitivity of the OS isn't important, 
> case sensitivity of the *file system* is. And the standard file system 
> on Mac OS, HFS+, defaults to case-preserving but case-insensitive.
> 
> (There is an option to turn case-sensitivity off, but hardly anyone uses 
> it because too many applications break.)

Oops, I meant to say there is an option to turn case-sensitivity ON, but 
hardly anyone uses it.


By the way, Linux users can experiment with case-insensitive file 
systems provided they have privileges to mount file systems. Here's an 
example. At the shell:


# allocate 100K for a (tiny) file system
dd if=/dev/zero of=fat.fs bs=1024 count=100

# format as FAT-12 with an optional label
/sbin/mkfs.vfat -n "FAT image" fat.fs

# mount it and create some files
sudo mount -o loop  fat.fs /mnt
sudo touch /mnt/{lower.txt,UPPER.TXT,mIxEd.TxT}


Now you have a file system and some empty files to experiment with.


-- 
Steven

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Case Insensitive Globing

[Steven D'Aprano]

On Sat, May 18, 2019 at 11:52:29AM +0100, Alan Gauld via Tutor wrote:
> On 18/05/2019 03:14, Richard Damon wrote:
> 
> > The same directory, running the same program under Mac OS X, which also
> > is a case insensitive file system, 
> 
> That is your mistake. Darwin, the core of the MacOS X system
> is a version of BSD Unix and like all Unix OS is very much
> case sensitive.

That's not quite right -- case sensitivity of the OS isn't important, 
case sensitivity of the *file system* is. And the standard file system 
on Mac OS, HFS+, defaults to case-preserving but case-insensitive.

(There is an option to turn case-sensitivity off, but hardly anyone uses 
it because too many applications break.)

https://stackoverflow.com/questions/4706215/mac-os-x-how-to-determine-if-filesystem-is-case-sensitive

https://apple.stackexchange.com/questions/71357/how-to-check-if-my-hd-is-case-sensitive-or-not


That means that, like Windows file systems FAT and NTFS, file names are 
case-insensitive: files "Foo", "foo" and "FOO" are all considered the 
same. But unlike Windows, the file system preserves the case of the file 
as you created it, so if you created it as "foO" that's how it will be 
recorded on the disk rather than normalizied to "foo".

Fun fact: even NTFS supports a case-sensitive mode! But again, hardly 
anyone uses it. Likewise, if you are using Samba on Unix/Linux, you can 
have case-insensitive file operations on an underlying case-sensitive 
file system.


> Some of the GUI tools in MacOS X may work as if they were case
> insensitive (presumably for backwards compatibility to MacOS 9)
> but the underlying OS is case sensitive and all the Terminal
> type tools and commands, including Python, follow suit.

I don't think that is correct:

https://apple.stackexchange.com/questions/22297/is-bash-in-osx-case-insensitive



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How arguments to the super() function works?

[Steven D'Aprano]

On Sat, May 18, 2019 at 09:51:39PM +0530, Arup Rakshit wrote:

> Here, why super(Role, self).__init__(**kwargs) is used instead of 
> super().__init__(**kwargs) ? What that Role and self argument is 
> instructing the super() ?

The Role and self arguments are the owning class and current instance. 
They are always required, but in Python 2 you needed to provide them 
yourself, and it was error-prone. So in Python 3, the interpreter was 
modified to do the right thing when super is called with no arguments.

super() still accepts manual arguments, and in fact the signature is 
quite complex:

class super(object)
 |  super() -> same as super(__class__, )
 |  super(type) -> unbound super object
 |  super(type, obj) -> bound super object; requires isinstance(obj, type)
 |  super(type, type2) -> bound super object; requires issubclass(type2, type)


but any use other than plain old super(Class, self) in Python 2 or 
super() in Python 3 is considered rather advanced.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Two Scripts, Same Commands, One Works, One Doesn't

[Steven D'Aprano]

Hi Stephen,

My responses are interleaved with your comments below. Please excuse the 
length of my post, but your problem is a complicated one.


On Wed, May 15, 2019 at 08:16:02AM -0400, Stephen P. Molnar wrote:
> I am writing scripts to semi-automate some of my Quantum Chemistry 
> software and have encountered a problem that has me baffled. The two 
> scripts have the same form, the only difference being the commands. One 
> script works, the other bombs.

A note on terminology: to programmers, the terminology "bombs" usually 
refers to a fatal application crash detected by the operating system:

https://en.wikipedia.org/wiki/Bomb_%28icon%29

whereas what you are experiencing is merely an exception with a 
traceback displayed. The difference is that a "bomb" (or a "crash") is a 
serious bug in the application (Python), while an exception is actually 
a sign that Python is working correctly.

The bottom line is that there's not much you can do about a bomb except 
stop running the script, whereas an exception means there's a bug in 
your code you can fix.


> The script that works is:
[...]

Thanks for the careful and detailed samples of your code, but in general 
unless you are paying a consultant to to the work for you, it is best to 
try to narrow down the samples to the smallest amount that demonstrates 
the problem. We're volunteers, donating our time for free, and often 
with very limited time to offer. Every extra line of code you present us 
with discourages us a little more from tackling your problem, so the 
smaller the amount of code, the more likely we are to help.

For a better explanation, you might like to read this:

http://www.sscce.org/

It is written for Java programmers but applies to any language.



> #!/usr/bin/env python

This line tells a Unix system to run the script using "python", which is 
probably going to be Python 2. But I see from your code below that you 
are actually running Python 3 code. So this is a potential problem. I 
cannot tell if it is *actually* a problem right now, or will only become 
one in the future.

You could try replacing the word "python" with "python3", but you should 
run these commands at the Unix shell first:

env python
env python3

and see what they say.

Another problem is that both of your scripts call out to an external 
script `pythonsh` which we cannot see, so there's no way of knowing what 
that is doing.

The second script refers to yet another Python script 
`prepare_pdf4.py` but the output says this file doesn't exist:

can't open file
'./prepare_pdf4.py': [Errno 2] No such file or directory


So that's a problem you need to fix. I have no idea if that will fix 
everything, or just part of the issue.



> while the script that bombs is:

[...]
> Traceback (most recent call last):
> 
>   File "", line 1, in 
> runfile('/home/comp/Apps/Models/1-NerveAgents/Ligands/calc_pdf.py', 
> wdir='/home/comp/Apps/Models/1-NerveAgents/Ligands')
> 
>   File 
> "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py",
>  
> line 824, in runfile
> execfile(filename, namespace)


This tells me that you are running your code via the Spyder IDE. That 
means you have *at least* five and maybe six components involved:


- the Python interpreter
- which runs the Spyder IDE
- which runs your script
- which runs the mystery file pythonsh;
- which does something (runs it?) with the prepare_ligands4.py 
  file or the (missing) prepare_pdf4.py file;
- which does who knows what.

When having problems debugging this code, it helps to simplify the 
problem by cutting Spyder out of the chain. If you run your script 
directly, using a command like

python3 path/to/the/script.py

does the error go away? I suspect not, but you can keep that in mind for 
future debugging.



>   File 
> "/home/comp/Apps/miniconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py",
>  
> line 110, in execfile
> exec(compile(f.read(), filename, 'exec'), namespace)
> 
>   File "/home/comp/Apps/Models/1-NerveAgents/Ligands/calc_pdf.py", line 
> 23, in 
> print ('Subprocess FAILED:', proc.command)
> 
> AttributeError: 'Popen' object has no attribute 'command'

That error message is completely correct: Popen objects don't have an 
attribute called "command".

You are tring to print your own error message, referring to 
"proc.command" but there is no such thing. What are you trying to do? 
You need to find another way to do it.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] self.name is calling the __set__ method of another class

[Steven D'Aprano]

On Tue, Apr 30, 2019 at 12:47:02AM +0530, Arup Rakshit wrote:

> I really didn't write that code by myself. The day I'll you will not see 
> me here everyday :) . I was watching a PyCon video 
> https://youtu.be/81S01c9zytE?t=8172 where the author used this code. But 
> his explanation is not clear to me. The main problem is that the guy who 
> was recorded it far away from the projector, so what speaker were 
> showing there is not clear. So thought to ask here as usual. Because I 
> felt so lost with this trick.

Okay, the short, SIMPLIFIED (and therefore inaccurate) summary of 
descriptors:

Descriptors are the "magic" used by Python whenever it does an attribute 
lookup. When you do any sort of attribute lookup or assignment:

x = spam.eggs

spam.eggs = value

Python looks at spam and spam's class for an attribute called "eggs", 
and if that attribute is an object with a __set__ or __get__ method, it 
calls that method:

x = spam.eggs
=> x = spam.eggs.__get__()

spam.eggs = value
=> spam.eggs.__set__(value)

For the gory details of what *precisely* happens, see the Howto Guide:

https://docs.python.org/3/howto/descriptor.html


Python has a few common descriptors built in:

- ordinary methods
- classmethod
- staticmethod
- property

Apart from staticmethod, they're all pretty common in code. But writing 
your own custom descriptors is fairly rare. I've only done it once, in 
25+ years of using Python.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] self.name is calling the __set__ method of another class

[Steven D'Aprano]

On Mon, Apr 29, 2019 at 11:25:51PM +0530, Arup Rakshit wrote:

> Now I am not getting how the __set__() method from NonBlank is being 
> called inside the __init__() method. Looks like some magic is going on 
> under the hood. Can anyone please explain this how self.name and 
> self.email assignment is called the __set__ from NonBlank? What is the 
> name of this concept?


I haven't read your code in detail, but it sounds like the Descriptor 
protocol. Descriptors are used "under the hood" by Python to implement 
methods, classmethod, staticmethod and property, among others, and are 
considered an advanced technique (only slightly less advanced than 
metaclasses).

https://docs.python.org/3/howto/descriptor.html

If you are *not* intentionally trying to write a custom descriptor, you 
should not use a __set__ method. (Perhaps you meant __setitem__?)

In general, you should treat all dunder (Double UNDERscore) methods as 
private to Python, and only implement those that you need. Don't use 
them for your own purposes.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] urlGET information

[Steven D'Aprano]

On Sun, Apr 28, 2019 at 03:00:12PM +1000, Nathan D'Elboux wrote:

> this is a learning exercise for me so how do more experienced python coders
> approach modules like this, when you encounter a module that has sparse
> documentation how do you learn more about the functionality of said module?
> 
> unpack it and just read the source?

If there's no documentation, read the source. If the documentation is 
inaccurate, incomplete or poor, read the source. If the source code is 
hard to read and understand, cry.

> or is there like a linux man page
> equivalent within the wheel that would provide more context?

Wheels don't magically create documentation out of thin air. If the 
library isn't documented, where would it get the documentation from?



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When you think to setup the __class__ of a module object to a subclass of ModuleType

[Steven D'Aprano]

On Fri, Apr 26, 2019 at 06:18:32PM +0530, Arup Rakshit wrote:

> BTW, one thing I would like to know about this list is that, everytime I 
> send an email I see it the in list after 2 hours approx. Is this for me 
> or everybody? I am just curious.

Just you, I think.

Looking at the headers of your post, I see when you sent the post:

Received: from arup.local ([117.194.106.70]) by smtp.gmail.com
 Fri, 26 Apr 2019 05:48:35 -0700 (PDT)

and then a few hops through google, to mail.python.org, and then there 
seems to be a long delay before mail.python.org sends it back out again.

Perhaps you are stuck in the moderator queue?



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When you think to setup the __class__ of a module object to a subclass of ModuleType

[Steven D'Aprano]

On Thu, Apr 25, 2019 at 02:52:07PM +0530, Arup Rakshit wrote:

> Here it is: *3.3.2.1. Customizing module attribute access* 
> (https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access)

Oh! That's brand new in 3.7, no wonder I didn't know about it.

I did see the core developers talking about adding this feature, but I 
didn't know that they had done so.

Your original question was:

> In the simple code like what are the advantages we get from? Is this 
> so that we can implement more special methods than just __getattr__ 
> and __dir__ in the module level?

Yes, that's what the documentation says. I don't see any reason not to 
believe it.

Oh, this is cool! I'm going to enjoy playing with this...


py> from types import ModuleType
py> class Magic(ModuleType):
... count = 0
... @property
... def spam(self):
... self.count += 1
... return ' '.join(['spam']*self.count)
...
py> import sys
py> sys.modules['__main__'].__class__ = Magic
py> import __main__
py> __main__.spam
'spam'
py> __main__.spam
'spam spam'
py> __main__.spam
'spam spam spam'



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] module import from a packager works inconsistent between REPL and command line

[Steven D'Aprano]

On Thu, Apr 25, 2019 at 05:40:18PM +0530, Arup Rakshit wrote:

> I have a small app like this:

Please simplify your code to the minimum needed to demonstrate the 
problem you are asking about. This bit is excellent:

> pizza-shop$ tree .
> .
> └── pizzapy
> ├── __init__.py
> ├── menu.py
> └── pizza.py
> 
> 1 directory, 3 files

Nicely shown!

But we don't need to see all the gory details of menu.py and especially 
not of pizza.py, all those methods in pizza.Pizza are irrelevant to the 
problem. Please read this:

http://www.sscce.org/

for a guide. It is written for Java programmers, but it applies to any 
language.

All we need in menu.py is a single line:

# menu.py 
from pizza import Pizza

because that's the line that fails.

And for pizza.py, all we need is:

# pizza.py 
Pizza = None


> Now when I call the menu.py from command like it works as expected.

When you ask Python to import a module, it doesn't search the entire 
hard drive, that could take hours if the drive is big enough. It only 
looks in the search-path. At runtime, you can see the paths searched 
like this:

import sys
print(sys.path)

which will show you where Python is looking for modules.

When you directly call a module:

python path/to/menu.py

the directory holding that module is included in the search path, so if 
you also have path/to/pizza.py the "import pizza" will work.

But in the REPL, only the default search path is used. 

In your case, the fix is, I think, to change menu.py to do this:

# was: from pizza import Pizza
from pizzapy.pizza import Pizza

which I *think* will solve the problem, but I haven't tested it.

For what it is worth, importing problems are sometimes annoying to 
solve. What works as a good design for importing libraries doesn't 
always make a good design for scripts that you call directly from the 
command line, and visa versa, so the import system is a bit of a 
compromise between the two.



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] str.replace error

[Steven D'Aprano]

On Thu, Apr 25, 2019 at 10:46:31AM -0700, Roger Lea Scherer wrote:

> with open('somefile') as csvDataFile:
> csvReader = csv.reader(csvDataFile)
> for row in range(100):
> a = "Basic P1"
> str.replace(a, "")
> print(next(csvReader))


I'm not quite sure what you expected this to do, but you've 
(deliberately? accidently?) run into one of the slightly advanced 
corners of Python: unbound methods. Apologies in advance, but this may 
be a bit technical. For a suggestion on how to fix the problem, see 
right at the end.

Normally when you call a string method, you do something like this:

mystring = "Something here"
newstring = mystring.replace("here", "there")

The technical name for the mystring.replace call is a "bound method", as 
the method is "bound" to a string instance (mystring). The method 
actually takes *three* arguments: the string to operate on, the 
substring to be replaced, and the replacement string:

   (string to operate on) . replace ( substring to be replaced, replacement )

But under the hood, what that turns into is an "unbound method" that 
calls the method attached to the *class* "str":

str.replace(string to operate on, substring to replace, replacement)

Alas, when you call an *unbound* method like this:

str.replace(a, "")

the interpreter may get a bit confused and report a slightly inaccurate 
error message:

> TypeError: replace() takes at least 2 arguments (1 given)

For technical reasons, all the counts are off by one. The error message 
should say:

TypeError: replace() takes at least 3 arguments (2 given)

but because the same error message gets used for both bound and unbound 
methods, the counts are off for unbound methods. This is an annoyance, 
but given that unbound methods are relatively rare, not a large one.

So how do you fix your code?

(1) First of all, you need a string to operate on. I'm not sure which 
string that should be -- do you want to check *every* cell? Just the 
first cell in each row? Every second row? That's up to you to decide.

(2) Secondly, strings in Python are "immutable" -- that is to say, they 
cannot be changed in place. Unlike languages like (say) Pascal, if you 
want to change the value of a string, you need to re-assign a new 
string:

a = "Hello World!"
a.upper()  # Doesn't work!!!
print(a)  # Still gives "Hello World!"

Instead you need to say a = a.upper().

(3) Unbound methods are a slightly advanced technique which you don't 
need here, so you should just use normal method call.



Putting those three changes together gives something like this:


with open('somefile') as csvDataFile:
csvReader = csv.reader(csvDataFile)
for row in range(100):  # How do you know there are only 100 rows?
mystring = "Something here? ~Basic P1~ but I don't know what"
print("Before:", mystring)
mystring = mystring.replace("Basic P1", "")
print("After:", mystring)
print(next(csvReader))



Hopefully that's enough to set you on the right track. If not, please 
feel free to write back to the list and ask more questions!




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] When you think to setup the __class__ of a module object to a subclass of ModuleType

[Steven D'Aprano]

On Thu, Apr 25, 2019 at 11:30:28AM +0530, Arup Rakshit wrote:
> On 25/04/19 3:52 AM, Alan Gauld via Tutor wrote:
> >On 24/04/2019 12:22, Arup Rakshit wrote:
> >>In the simple code like what are the advantages we get from?
> >I'm not really sure what you are asking about?
>
> Ok. My question is that when people add such a class to the module? What 
> is the goal/intention they met by setting such __class__ attribute of a 
> module. The documentation I am reading just shown this technique but 
> didn't dig into this. They didn't explain why/when should we use this 
> technique, but the how.

Can to share which documentation you are reading? I've been using Python 
since version 1.5, and I've never heard of this technique.

Thanks.

-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What protocol to follow when need to pick either one from __getattr__ and __getattribute__ ?

[Steven D'Aprano]

On Tue, Apr 23, 2019 at 08:22:56PM +0530, Arup Rakshit wrote:
> I read today 2 methods regarding the customizing the attribute 
> access:__getattr__ and __getattribute__ from 
> https://docs.python.org/3/reference/datamodel.html#special-method-names. 
> What I understood about them is that __getattr__ is called when the 
> requested attribute is not found, and an AttributeError is raised. But 
> later is called everytime unconditionally.


When you overload __getattribute__, your class will be slow because 
**every** attribute lookup has to go through your method, instead of 
only the lookups which fail. And unless you are very careful, you will 
break things:



py> class X(object):
... def __init__(self, x):
... self.x = x
... def __getattribute__(self, name):
... if name is 'x': return self.x
...
py> obj = X(999)
py> obj.x
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in __getattribute__
  [ previous line repeats 332 times ]
RecursionError: maximum recursion depth exceeded while calling a Python 
object



Normally, overriding __getattribute__ is considered a very advanced and 
unusual thing to do.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question on implmenting __getitem__ on custom classes

[Steven D'Aprano]

On Tue, Apr 23, 2019 at 08:27:15PM +0530, Arup Rakshit wrote:

> >You probably want:
> >
> >      def __init__(self, list=None):
> >  if list is None:
> >      list = []
> >  self.list = list
> 
> That is really a new thing to me. I didn't know. Why list=None in the 
> parameter list is different than in the body of __init__ method? Can you 
> elaborate this?

Try running this code and see what happens:

def make_default():
print("creating a new list")
return []


def function(arg=make_default()):
arg.append(1)
return arg


Now call:


function()
function()
function()


and see if you can work out what is happening.

Hint: how many new lists are created? when are they created? 



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question on implmenting __getitem__ on custom classes

[Steven D'Aprano]

On Tue, Apr 23, 2019 at 11:46:58AM +0530, Arup Rakshit wrote:
> Hi,
> 
> I wrote below 2 classes to explore how __getitem__(self,k) works in 
> conjuection with list subscriptions. Both code works. Now my questions 
> which way the community encourages more in Python: if isinstance(key, 
> slice): or if type(key) == slice: ?

In general, we should normally use `isinstance`, because it works with 
subclasses.

But `slice` can't be subclassed:

py> class S(slice):
... pass
...
Traceback (most recent call last):
  File "", line 1, in 
TypeError: type 'slice' is not an acceptable base type


so there is no advantage to using `isinstance`. (There is no 
disadvantage either.)

I would use `type(key) is slice` to guarantee[1] that the key is 
certainly a slice.

Why use `is` instead of `==`?

The `is` operator will be a tiny bit faster than `==`, but more 
importantly, you could have a class designed to pretend to be a slice. 
It isn't easy to do (you would have to write a metaclass, which makes it 
an advanced technique) but by using `is` we can eliminate even that slim 
chance.


> How should I implement this if I 
> follow duck typing, because none of the code currently I wrote using 
> duck typing techiniqe IMO.


Why bother? Duck-typing is good for *data* values, but a slice is not a 
data value, it is a way of specifying a range of indexes.


> class MyCustomList:
>     def __init__(self, list = []):
>     self.list = list

Watch out here, you have a mutable default value, that probably doesn't 
work the way you expect. The default value is created ONCE, and then 
shared, so if you do this:

a = MyCustomList()  # Use the default list.
b = MyCustomList()  # Shares the same default list!
a.append(1)
print(b.list)
# prints [1]

You probably want:

     def __init__(self, list=None):
 if list is None:
     list = []
 self.list = list


>     def __getitem__(self, key):
>     if isinstance(key, slice):
>     return self.list[key]
>     else:
>     return self.list[key]


The "isinstance" check is useless, because you do precisely the same 
thing in both branches.

    def __getitem__(self, key):
 return self.list[key]


will do exactly the same, and more efficiently.






[1] Not actually a guarantee.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Trouble with SUM()

[Steven D'Aprano]

On Sat, Apr 20, 2019 at 04:51:11PM -0400, Ju Bo wrote:

> con = "y"
> while con == "y":
> AMT = float(input("What is the price of the item? $"))
> con = input("Would you like to continue? [y/n]")
> price = float(sum(AMT + AMT))

That's a good first attempt. 

Rather than set the price to float(sum(AMT + AMT)), which doesn't work, 
set the price to 0 before the loop starts. Then each time through the 
loop, set the price to price + AMT:

price = price + AMT


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Mutable objects as dictionary keys: Accessing , Sorting and Using dictionary

[Steven D'Aprano]

On Thu, Apr 18, 2019 at 11:18:02AM -0400, Pushkar vaity wrote:
> Hi,
> 
> I am using Python 3.7 with anaconda install in PyCharm
> I am trying to traverse some directories using the os module and grab some
> tcl procedures and put the proc names in python dictionary as keys. Values
> are some other info related to the proc.
> 
> Problem: procs found in the directories can have similar names. But python
> dict keys cannot be same.

Sorry, can you explain why this is a problem? *Similar* is not 
enough to cause keys to clash. If you have a proc name "abcd" and a 
similar name "abcd.1", they count as different keys.

Could you use the full pathname?

"/some/directory/proc.123"

"/another/directory/proc.123"

would count as different keys.


> So I am using the mutable objects as the key.
> 
> For Ex:   class MyClass:
> 
> def __init__(self, name):
> self.name = name
> 
> def __repr__(self):
> return repr(self.name)

I'm not convinced that you actually need this class, but if you do, you 
ought to define a hash method:

def __hash__(self):
return hash(self.name)

and equality:

def __eq__(self, other):
if isinstance(other, MyClass):
return self.name == other.name
return NotImplemented


otherwise dict lookups will be by object identity, not value.


> proc_dict[MyClass(proc_name)] = (full_file, proc_args)
> 
> This has allowed me to have same separate keys in my dictionary.
> But now,
> 
>1. I am not able to access individual dictionary values by
> specifying only the key name.

Add the __hash__ and __eq__ methods as above, and then try:


values = proc_dict[ MyClass(proc_name) ]


>2. I am not able to use the 'in' or 'not in' operations on my
> dictionary successfully.

Again, you need __hash__ and __eq__ for them to work correctly.


>3. Also, I am not able to use the sorted() function on my dict
> items. It gives me an error as follows:
> 
> 
>- for proc_name, (file_name, proc_args) in sorted(proc_dict.items()):
>   - TypeError: '<' not supported between instances of 'MyClass'
> and 'MyClass'

You need to define a __lt__ method:

def __lt__(self, other):
if isinstance(other, MyClass):
return self.name < other.name
return NotImplemented


But again, I'm not convinced that you need this class at all. Can you 
please show us the proc names you get, how you get them, and what you do 
with them?

Using a wrapper class like this is probably the last resort.



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Questions about the deprecation of standard library modules

[Steven D'Aprano]

On Sat, Mar 30, 2019 at 10:13:16PM -0500, boB Stepp wrote:

> So my main question is how does one know in which Python version a
> deprecated module will be removed?  I'm not too concerned about the
> imp module, but _do_ want to know how the removal process works for
> deprecated standard library modules that I might be interested in.

The usual process goes something like this:

- after much discussion, the core developers agree to deprecate 
  a module;
- in a fit on enthusiasm, the docs are adjusted to note that the
  module is deprecated;
- many years later, somebody remembers the deprecation note in
  the docs, and wonders whether or not the module should finally
  be removed;
- the module is changed to give a warning when used;
- some years after that, somebody asks what's going on with
  the deprecated module, shouldn't it be removed by now?
- and the module is finally removed.

(Only half joking.)

In practice, it depends on the specific developer, and how good he or 
she is at coming back to the module and removing it after a "suitable" 
time. Suitable usually means a minimum of two releases, but it could be 
as little as one in urgent cases; in *very special circumstances* a 
module which is a security risk could be removed immediately, but that's 
rare.

Its not unusual for everyone to forget to remove a deprecated feature or 
module until long after it should have been removed, or to decide that 
deprecation notice or not, too many people are still using it and so it 
can't be removed.

On the other hand, its also not ususual for modules to be removed 
promptly. Usually its:

- one or two releases silently marked as deprecation pending;
- one or two releases marked depreciation (active warning);
- and then removed.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] error message

[Steven D'Aprano]

> I don't understand this error message. Thank you so much, Glenn Dickerson
> 
> Traceback (most recent call last):
>   File "/home/glen/app.py", line 1, in 
> import Student
>   File "/home/glen/Student.py", line 2
> def__init__(self, name, major, gpa, is_on_probation):
> ^
> SyntaxError: invalid syntax


Syntax errors are sometimes the hardest to decipher, because the message 
is usually pretty generic and uninformative, and the caret ^ will appear 
where the interpreter *notices* the problem, not where the problem 
*starts*.

In this case, the problem is you are missing a space between the "def" 
keyword and the "__init__" method name.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Steven D'Aprano]

Hi Glenn, and welcome.

On Fri, Mar 15, 2019 at 09:54:41PM -0400, Glenn Dickerson wrote:
> class Student():
> def__init__(self, name, major, gpa, is_on_probation):
> self.name = name
> self.major = major
> self.gpa = gpa
> self.is_on_probation = is_on_probation
> 
> 
> import Student
> student1 = Student('Jim', 'Business', 3.1, False)
> student2 = Student('Pam', 'Art', 2.5, True)
> print(student1.name)
> print(student2.gpa)
> 
> I entered this in IDLE and it failed. Any advice? Thank you.

Yes -- read the error message. What does it say?


(Reading, and understanding, error messages is probably the most 
important skill a programmer can have.)



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Regular Expressions (Re.sub) Function

[Steven D'Aprano]

Hi Edward, and welcome.

Please remember that we're volunteers, doing this for free. Unless your 
problem is really interesting, you're not likely to get people 
volunteering to spend a long time slogging through multiple attachments, 
screenshots, at least five seperate attempts, etc.

By the way, most of your attachments have been deleted by the mailing 
list. Only things which it considers text files will get through.

Please try to simplify the question you are asking, and keep the code 
you show as short and simple as practical. You might find it helpful to 
read this:

http://www.sscce.org/

In the meantime, this might help you:

py> import re
py> text = "abcd[][]...[x]yz"
py> re.sub(r'\[|\]', '@', text)
'abcd...@x@yz'
py> re.sub(r'\[|\]', '', text)
'abcd...xyz'






On Mon, Mar 04, 2019 at 11:04:15AM +0300, Edward Kanja wrote:
> Hi there,
> Hope this finds you well, I'm working on a simple project to analyse data
> using
> regular expressions. I have successfully worked on a regular expression
> pattern
> that extracts data from my txt file. Unfortunately after i extract the data
> my output has too much of square brackets and by so doing the output cant
> be well exported in a csv file. I have worked on an idea of Re.sub function
> where i'm trying to replace all square brackets in my output with an empty
> string.
> Below are images of what i have been working on.
> 
> Explanations of the attachments and images named as follows:
> *1. mydata in txtfile. *This images shows how my data looks in the txt file
> although
> i have also attached the real txt file with the actual data below.
> *2. code for the output. *This image shows the regular expression that is
> well extracting the data from my txt file.
> 3. *my output. *This image shows the out after i ran my code . it shows the
> square brackets that are in my output and ones i need to do away with.
> 4.*tryone code. *This image shows my first try using re.sub to eliminate
> the square brackets.
> *5. tryone output.*This images shows the output i had after running the
> code in the image named tryone code.
> *6. try two code:*This image shows my second try using re.sub to eliminate
> the square brackets.
> *7.trytwo output.*This images shows the output i had after running the code
> in the image named trytwo code.
> *8.try four code.*This image shows my fourth try using re.sub to eliminate
> the square brackets.
> *9.try four output.* This images shows the output i had after running the
> code in the image named tryfour code.
> *10.try five code.*This image shows my number five try using re.sub to
> eliminate the square brackets
> *11.*  *try five output.*This images shows the output i had after running
> the code in the image named tryfive code.
> *12.line five name code regex.*This image shows the full code for the name
> regular expression with the re.sub function for the code
> in the image named try five code.
> *13.try six code.*This image shows my sixth try using re.sub to eliminate
> the square brackets
> *14. try six output.*This images shows the output i had after running the
> code in the image named tryfsix code
> *15.try six full name code.*This image shows the full code for the name
> regular expression with the re.sub function for the code
> in the image named try six code.
> *16. unon.TXT. *This is the txt file that has the data i am extracting.
> 
> Kindly assist and kindly copy my work email ( edward.mwa...@un.org ) when
> replying .
> Thanks alot.
> 
> Regards,
> Kanja Edward.
> P.O.BOX 1203-00300,
> NAIROBI.
> *+254720724793*
> www.linkedin.com/in/edward-kanja-bba16a106
> 

> --
> |Rawzeea NLKPP | VE11-Nairobi 
>| 20002254-MADIZ| 00   | 00   
> |Regular Scheme B | 15-JAN-2019 To 31-DEC-2019 | No   |
> ||
> | 3648 | 18-FEB-2020 | Yes | 03   
>|  .00 USD  | 0.00 USD  | Leased Commercially  
>  |   00.00 KES| No   |
> ||
> | R-9-01   | 00.00%  |0023.28 KES  |000.18 
> USD   |   |  1.0  |  0.00|  0.00  
>   | No  | 10-JAN-2019 To 31-DEC-2019|
> 

Re: [Tutor] schedulers

[Steven D'Aprano]

Sorry, I hit send too quick...

I realise you haven't written a busy loop, but the way you wrote your 
scheduler, you can only schedule jobs to the nearest 5 seconds. That's 
probably okay for a backup process that might take a few hours to run, 
but it is hardly a good solution to things that might need to be 
schedules 1 or 2 seconds apart.

And while you aren't *continuously* checking the time, you're still 
checking it pretty often. In the course of a day, you check the 
time 17280 times to see if you should run your backup program. That's 
pretty busy for something running once a day :-)

There's also the problem that you have to remember to launch your 
scheduler program every time you log in. Again, registering your backup 
program with the OS once means you don't have to think about it.

-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] schedulers

[Steven D'Aprano]

On Thu, Feb 28, 2019 at 02:45:59PM +, nathan tech wrote:
> Hi there,
> 
> I recently started working on a backup program, and the one big feature 
> everyone wants in backup programs is the ability to schedule backups, right?
> 
> but I'm thinking, should I do this?
[...]
> Is that wise? Is that how it should be done?

No. You should write your backup program to do backups, and then 
register that program with your operating system's scheduler. I don't 
know what that's called on Windows, but on Linux that would be "cron".

Basically, you have three problems:

1. Perform the backup.

2. Let the user decide when to do the backup.

3. Actually trigger the program to run at the correct time.


Parts 1 and 2 are your job. Number 3 is up to the OS. It will handle all 
the complex problems of clocks changing between jobs, computer reboots 
(what if the computer is off when the job is supposed to run?) etc.

If for some reason you need to write your own scheduler, using a "busy 
loop" where you run a loop continually checking the time is not the 
right answer. (I don't know what the right answer is, I just know that 
the OS has already solved that.)



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] I flip a coin 1 million times what is the consecutive times it will come up head and come up tails

[Steven D'Aprano]

On Tue, Feb 26, 2019 at 01:34:33PM -0700, shaef...@q.com wrote:

> I am learning python.  I wanted to test my ability by making a program to
> keep track of the flip of a coin to find how many consecutive times it came
> up heads and tails.

Here's how to count the number of times it comes up heads (1) or tails 
(0):

from collections import Counter
from random import randint
c = Counter(randint(0, 1) for i in range(100))
print(c)

Obviously when you do it you will get different results, but here's 
mine:

Counter({1: 500481, 0: 499519})


If you want to count how many *consecutive* heads or tails happen, we 
can do this:

from itertools import groupby
it = groupby(randint(0, 1) for i in range(100))
c = Counter(len(tuple(x[1])) for x in it)
print(c)


When I run it, I get this:

Counter({1: 250180, 2: 124648, 3: 62760, 4: 31075, 5: 15707, 6: 7998, 7: 
3778, 8: 1966, 9: 989, 10: 501, 11: 219, 12: 136, 13: 49, 14: 22, 15: 
11, 16: 6, 18: 4, 17: 1})


which tells me that there are 250180 sequences of one head or one tail; 
124648 sequences of two heads or two tails; 62760 sequences of three 
heads or three tails, and so on to 4 sequences of 18 heads or tails.

If you want to see what is going on a bit better:

it = groupby(randint(0, 1) for i in range(20)) # a bit smaller...
for k, x in it:
t = tuple(x)
print("got", len(t), "consecutive", ("heads" if k==1 else "tails"), t)


which when I ran it gave me:

got 1 consecutive heads (1,)
got 1 consecutive tails (0,)
got 4 consecutive heads (1, 1, 1, 1)
got 3 consecutive tails (0, 0, 0)
got 1 consecutive heads (1,)
got 1 consecutive tails (0,)
got 7 consecutive heads (1, 1, 1, 1, 1, 1, 1)
got 1 consecutive tails (0,)
got 1 consecutive heads (1,)




-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Only appending one object to list, when I am expecting more than 1

[Steven D'Aprano]

On Tue, Feb 26, 2019 at 09:09:56AM +, AdamC wrote:

> def createObjects(f):
> '''Takes a file object and iterates through entries, passing them to
> create
> object, depending on what object it is.'''
> for line in f:
> count = count + 1

This cannot be the code you are actually using, because that raises 
UnboundLocalError. count is never initialised, so that function you give 
cannot possibly run as shown.


py> def test():
... count = count + 1
...
py> test()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2, in test
UnboundLocalError: local variable 'count' referenced before assignment


There's no point us trying to debug code you aren't actually running. 
Try again with the actual working code. It might help if you read this:

http://www.sscce.org/



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Proxy for Python?

[Steven D'Aprano]

On Wed, Feb 06, 2019 at 02:03:01PM +, beech 48 wrote:

> Hi i am totally new to this, i am not a coder and am lost.

We are coders and we are still lost, because we can't read your mind and 
we have no idea what you are talking about.

Instagram? How is that relevant?

You say you are entering a proxy, but *where* are you entering it and 
what are you doing with it?



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ssl verification failed error

[Steven D'Aprano]

On Thu, Jan 24, 2019 at 03:03:28PM +1100, Riley Harris wrote:

> When I try to install packages in python this error occurs.

Please copy and paste (don't summarise or re-type from memory) the 
actual command you are running and the exact error message you receive.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Function not using updated variable?

[Steven D'Aprano]

Hi Ello, and welcome!

On Mon, Jan 21, 2019 at 01:14:55AM +0100, Ello Solcraft wrote:

> # currLocation doesn't use the updated variable.

How do you know?

I'm sorry, but it is impossible for me to tell what is going on here. 
There simply isn't enough information to understand your code.

We don't know what inputList, currCalc, mapLocation or mapInput do. We 
don't know how you know that currLocation doesn't "use the updated 
variable" (currCalc2), or what you mean by "use".

When asking for help to solve a problem like this, it is very helpful if 
you can provide SIMPLE code we can run ourselves, as well as the results 
you expect and the results you actually get.

(We're volunteers, and you are not paying us to troll through 200 or 300 
lines of code looking for a bug. Keep it short and simple.) 

More comments below.

> for x in inputList:
> currCalc.append(x)
> currCalc2 = ''.join(currCalc)
> currLocation = mapLocation(mapInput(currCalc2))#mapInput(currCalc2))
> 
> # I tried printing currCalc2, it updates like it should. But currLocation
> doesn't use the updated currCalc2. 

How do you know? What happens if you print(currLocation) each time 
through the loop?

What do mapInput and mapLocation do? If currLocation never changes its 
value, perhaps one of mapInput or mapLocation always returns the same 
thing?



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Implementation of list comparison operators

[Steven D'Aprano]

On Thu, Jan 17, 2019 at 03:05:17PM -0600, David Rock wrote:

> In [7]: nan == nan
> Out[7]: False
> 
> In [8]: a = 1.1
> 
> In [9]: a ==a
> Out[9]: True


> both a and nan are floats, so why does a == a work, but nan == nan 
> doesn’t?

They both "work", because they both do what they are designed to do.

Equality between two floats works something like this:

if either number is a NAN:
return False
if both numbers are 0.0 or -0.0:
return True
if both numbers have the same bit-pattern:
return True
otherwise return False


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Re: Doubt in Python

[Steven D'Aprano]

On Thu, Jan 17, 2019 at 09:57:03AM +, Alan Gauld via Tutor wrote:

> The algorithm is probably described somewhere in the documentation
> but my understanding is that it looks something like this(in pdeudo code):

List, tuple and string comparisons are defined as lexicographical order:

http://docs.python.org/tutorial/datastructures.html#comparing-sequences-and-other-types

https://en.wikipedia.org/wiki/Lexicographical_order

That's a fancy way of saying "dictionary order". That means that lists 
are ordered in the same way that words are ordered in the dictionary:

- match up letters in the word (items in the list) in pairs;

- so long as the pairs of letters (items) are equal, keep going;

- as soon as you hit a pair that aren't equal, the order of that pair 
determines the order of the words (lists);

- if one word runs out of letters (list runs out of items), then it 
comes first;

- if all the pairs are equal, the words (lists) are equal.


Some examples:

[0, 1, 2] < [1, 2, 3] because 0 < 1

[0, 1, 2] < [0, 1, 2, 3] because the first three items are equal 
but the first list is shorter.

[0, 1, 2, 3, 4, 5] < [0, 1, 999] because 2 < 999

[0, 999, 999, 999] < [1, 2] because 0 < 1


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Debugging a sort error.

[Steven D'Aprano]

On Sun, Jan 13, 2019 at 01:16:10PM +1100, mhysnm1...@gmail.com wrote:

> Issue, following error is generated after trying to sort a list of strings.
> 
> description.sort()
> TypeError: unorderable types: float() < str()

That tells you that you don't have a list of strings. You have a list of 
strings with at least one float mixed in.


> There is over 2000 such entries. This used to work and now doesn't. Only
> change to the code was modifying the data variable from a list to a
> dictionary.

I doubt that. Anytime people say "nothing changed except..." it 
invariably turns out to be "oh yeah, these other ten things changed 
too..." 

*wink*



> Below is the full code:

Not it isn't, because the code you supply has at least one syntax error. 
So it is impossible for it to be the actual code you are running.

In any case, we are volunteers, not paid consultants. If you want to pay 
us $200 an hour to debug your code, you get to dump a big ball of mud in 
our laps and say "fix it". But as volunteers, we have no obligation to 
trawl through your code trying to debug it (unless it looks interesting, 
or easy, or we're bored and looking for something to do...)

To maximize your chances of people actually helping, please read this:

http://www.sscce.org/


Having said that, I have spotted one obvious bug:

> description = data['Description']
> 
> for i in description:
>   if not str(i):
> print "not a string")

(See what I mean about a syntax error?)

That does not do what you think it does. My *guess* is that you are 
trying to check that every item in description is a string. But that's 
not what you do: you run through the items, make a string copy of each 
one (regardless of what kind of object it is) and then test whether the 
string is not the empty string (which it never will be, unless the 
original was the empty string).

Instead, try this:

for item in description:
if not isinstance(item, str):
print(type(item), repr(item), "is not a string")



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Best way to write this countdown code

[Steven D'Aprano]

On Sat, Jan 12, 2019 at 09:44:21AM -0600, Joseph Gulizia wrote:
> Thanks in advance as I've gotten wordy.
> 
> I want to integrate the following working code into a website:
[...]

Start with a simpler question: how would you integrate *any* Python code 
into a website?

Few (i.e. no) browsers directly support Python. This is why nearly all 
client-side (running in the brower) web development is done in 
Javascript. Server-side code can be written in Python, or any other 
language (Perl and PHP are other common choices) but for code that runs 
in the browser, it is difficult to use anything but Javascript.

But all is not quite lost:

http://www.skulpt.org/

https://duckduckgo.com/?q=python+in+the+browser


Start by getting a simple "Hello World" type program running in the 
browser, and then move on to something more ambitious.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Doubt

[Steven D'Aprano]

On Mon, Jan 07, 2019 at 09:59:31PM +0530, Amit Yadav wrote:
> How can simply typing
> 
>  print "hello world"
> 
> work?
> Like without including any header file or import statements how can it work.

Why shouldn't it work? Python is not C and doesn't use header files.

In Python, the interpreter knows the meaning of print, just as it knows 
the meaning of strings ("abc") and ints (123) and floats (4.56).

Some operations (keywords, operators like + - * etc, a handful of 
special constants like None) are known to the interpreter. Other 
functions (len, chr, ord, min, max etc) are in the "builtins" module, 
which doesn't need to be imported because it is always available. And 
some functions are in additional modules which need to be imported.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python installtion

[Steven D'Aprano]

On Mon, Jan 07, 2019 at 03:36:01PM +0530, mousumi sahu wrote:
> Dear Sir,
> I am trying to install python 2.7.10 on HPC. Python 2.6 has already been
> install on root. I do not have root authority. Please suggest me how can I
> do this.

What's HPC?

If you don't have root permission, do you have permission to install 
anything?

Are you comfortable with installing from source? You will need a 
C compiler. If everything works, it is easy, but if there are problems, 
it be difficult or impossible to solve them.

Would it be easier to just ask your system administrator to install it 
for you?


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] running a game server

[Steven D'Aprano]

On Sun, Jan 06, 2019 at 02:14:10PM +, nathan tech wrote:

> My question is, is python really the way to go for game servers?

*shrug*

Do game servers have unique requirements that are different from (say) 
mailing list servers and other servers? That's not a rhetorical 
question. I don't know the answer.

My home server runs at least three long-running Python processes (hpssd, 
whatever that is, fail2ban, and mailman), and I frequently run 
interactive Python sessions which stay open for a week at a time. It has 
been up for 62 days now, and the only reason that uptime isn't three 
times that is because I had a power failure. Now admittedly it is a home 
server with two users, not a public game server with 100,000 users, but 
still, this is evidence that Python does not leak memory.

You might consider that Stackless is used as the game scripting engine 
for "Eve Online", and they do have hundreds of thousands of users.

(Stackless is a version of Python that avoids the C stack.)


> I remember, long ago, running a script that went along the lines of:
> 
>      import shutil
>      import time
>      while 1:
>   f=open("backup.count_file.txt","r");
>   num=int(f.read())
>   f.close()
>   f=open("backup/count_file.txt","w");
>   f.write(str(num+1));
>   f.close()
>   shutil.copy("my_file.txt", "my_file.txt.backup."+str(num))
>   time.sleep(900)
> 
> 
> After running for a day, this had to be killed because it sucked up more 
> memory than anything I'd ever seen before.

As Alan says, it is hard to debug code that may or may not be the actual 
code you ran, and there's no obvious reason why your code should leak.

The first improvement I would make (aside from fixing the obvious typos, 
removing unnecessary semicolons, etc) is that there's no need to read 
the count from a file on every backup. You only need to read it once, 
when the script starts:


import shutil
import time
f = open("backup.count_file.txt", "r")
num = int(f.read() or '0')
f.close()
while True:
num += 1
    f = open("backup.count_file.txt", "w")
    f.write(str(num))
    f.close()
    shutil.copy("my_file.txt", "my_file.txt.backup." + str(num))
    time.sleep(900)


I suppose it is possible that shutil.copy has, or had, a memory leak, 
but without knowing the actual code you used, the version of Python, 
and the platform, I don't propose trying to investigate any further.

[...]
> I don't mind too much if its using a bit of memory, but, if python is 
> just going to leak all over the place and I'm better off learning c or c 
> sharp or something, then I'd like to know!

So you're a game developer, but you don't know Python, C, C# or 
"something". What language(s) do you know?

P.S. please reply to the mailing list, not me personally.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] function question

[Steven D'Aprano]

Hello David, and welcome!

On Sat, Jan 05, 2019 at 11:18:04AM -0500, David Lynch wrote:

[...]
> From what I've read about functions I should be able to define a function
> with 2 variables? And then I can add all of my code into that function by
> indenting it. 

So far so good! Here's an example of such a function:


def add(x, y):
return x + y


Of course in real programs we wouldn't bother with such a trivial 
function, since it is easier to read and write "a + b" rather than 
"add(a, b)". But it illustrates the concept.


> And then shouldn't I be able to
> function(float(input("enter input: ")))
> function(float(input("enter input 2: ")))
> return(function)
> ??

No, nothing like this. You are calling the function twice, but each time 
only supplying a single argument when it needs two. And the "return" 
keyword is only legal inside functions.

I *strongly* recommend that you make use of one of Python's best and 
most powerful features: the interactive interpreter. There you can try 
out small snippets of code with ease, see the result of running code 
live, use Python as a calculator, etc. That's an excellent way of 
learning to use the language, and you can always ask for help here if 
you have questions.

Do you know how to start the interactive interpreter? That will depend 
on your operating system (Windows, Mac, Linux) but generally you will 
open a "terminal" or "console", which gives you a command-line. That 
will have a % or $ prompt where you type commands.

Type "python" and hit the ENTER key. You will then see something similar 
to this:

  Python 3.5.2 (default, Oct 12 2016, 10:47:40)
  [GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux
  Type "help", "copyright", "credits" or "license" for more information.

and then be presented with >>> the default Python prompt, where you type 
Python code. In my examples below, I use the custom prompt "py>" because 
I think it looks nicer :-)

If you can get to the point of seeing the Python prompt, you're ready to 
start running live Python code. At the prompt, type:

12*23

and hit ENTER, and you should see the answer 276:

py> 12*23
276


Now type in your function. Notice that you need to indent the body of 
the function, and that the prompt changes to ... inside the body.


py> def add(x, y):
... return x + y
...
py>


(Remember, don't type the "py>" or "..." prompts.)

Now you should be ready to try calling the function:


py> add(99, 7)
106


If you get to that point, then you're ready to go to the next stage: 
intentionally doing something wrong so you can see what errors to 
expect.

What happens if you enter:

add(99)

at the prompt? What error do you get?

Once you've understood that, they you can start writing more serious, 
useful functions.

Good luck, and don't hesitate to ask if you have any further questions!



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Log file for Nested if-elif

[Steven D'Aprano]

On Wed, Jan 02, 2019 at 08:39:53PM +0530, Asad wrote:
> Hi All ,
> 
> Need advice on the following piece of code :

Let me write it in a more "Pythonic" style:



PATCH = r'\?/patch/\d{8}/\d{8}/admin/load.sql'
APPLY = r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}'
ERROR = r'set_metadata'

tail = deque(maxlen=8)  # the last eight lines
script = None
with open("file1.log", 'r') as f:
for line in f:
tail.append(line)
if (re.search(PATCH, line, re.IGNORECASE) 
or re.search(APPLY, line, re.IGNORECASE):
script = line
elif re.search(ERROR, line, re.IGNORECASE):
 print "Reason for error \n", line
 print "Script:", script
 print "Tail:\n", tail
 print " Danger "  # Seriously? This is dangerous?
 break


> Now this is printing the last cached line in the variable line   . However
> I would like to see the following output :
> 
> 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then
> look for the line "set_metadata" in the file1.log  if it finds the pattern
> then print the line which matches the pattern
> \?/patch/\d{8}/\d{8}/admin/load.sql

Where are you looking? *Anywhere* in the log file? Only in the 
immediate next line? Anywhere forward of the "patch" line? You are 
looking for two patterns, but does the order they appear, or the 
distance apart, matter?


blah blah blah set_metadata blah blah
... 100 lines ...
blah blah blah /patch/ ... admin/load.sql
... 100 lines ...

Is that a match?

> print last 4 lines of the tail array  and exit

print tail[-4:]
sys.exit()


 
> 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql'
> 
> then look of the anothern pattern
> :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern
> 
> then look for line "set_metadata" in the file1.log  if it finds the pattern
> then print the line which matches the pattern
> \?/patch/\d{8}/\d{8}/admin/load.sql

This is a contradiction: you just said that it DOESN'T match the 
patch...load.sql pattern, but now you want it to print the line that 
matched. There is no line that matches!

> 3 ) if it doesnot match  the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql
> or '\?/patch/\d{8}/\d{8}/admin/load.sql'
> 
> print "No match found refer to install guide"
> 
> Can you advice what I can do to change the code .



You need to tighten the specifications to make it more clear what you 
are trying to do.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Defining variable arguments in a function in python

[Steven D'Aprano]

On Sun, Dec 30, 2018 at 11:07:19AM -0500, Avi Gross wrote:
> Steve,
> 
> I had the same thoughts and many more when I played with these ideas 
> last night.

Pity that one of those thoughts wasn't "I shouldn't suggest a bad 
solution on a mailing list populated by beginners who won't recognise 
how bad it is". That would have saved both you and me a lot of time.

> I thought I stated clearly that this was an EXPLORATION 

And I explored the problems with the design.

[...]
> Consider a NORMAL function with no external gimmicks. Say it 
> accepts an argument (positional or by name does not matter) and 
> processes it by checking they argument type, the specific values 
> against a range expected, and so on. It then unilaterally makes 
> changes. If it expects a single value of type integer, ang gets a 
> floating point number, it may round or truncate it back to an integer.
[...]

Sure. But in general, such a design applies a known transformation of 
input argument to value actually used:

- any number -> ignore any fraction part;
- any string -> ignore leading and trailing whitespace;
- any string -> treat as case-insensitive;
- list of values -> sorted list of values;

which is not the same replacing out-of-band values with some default 
value. In each of the above examples, there is a known, direct 
connection between the caller's argument, and the value used:

- the argument, ignoring any fraction part;
- the argument, ignoring leading and trailing whitespace;
- the argument, ignoring case differences;
- the argument, ignoring the original order of items;

rather than:

- throw away the argument, and use an unrelated default value.

The closest analogue would be clamping values to a given size, so 
that out-of-band values are truncated to the outermost value:

clamp(999, min=1, max=10)
=> returns 10

but even that should be used with care. And as I said, it is acceptable 
to use an dedicated sentinel value like None or Ellipsis.

But using otherwise valid-looking, but out of range, values as a trigger 
for the default should be used with care, if at all.

Some designs are better (more likely to be useful, less likely to lead 
to bugs and surprises) than others.


> If your choice is in some sense wrong, or just not convenient, it 
> replaces it with another choice. 

That would very likely be a poor or dangerous design. If your argument 
is "wrong", that's an error, and errors should not in general pass 
silently.


[...]
> Again, discussing a different scenario. Would you agree that kind of 
> design does happen and sometimes is seen as a smart thing but also as 
> full of pitfalls even sometimes for the wary?

Sure, it does happen, often badly. 


> In that spirit, any design like the ones I played with is equally 
> scary. Even worse, I have a QUESTION. Let me remind readers of the 
> initial idea behind this. You have a function in which you want to 
> communicate with the python ruler that it have a positional argument 
> that will be switched to a default. But the current rules are that the 
> only way to make something optional is by making it a keyword 
> parameter and declare the default right there.

No, that's not the current rules. Positional values can have default 
values too. You just can't skip positional values: there's no way of 
saying "I'm not supplying argument 1, but I am supplying argument 2" 
using only positional arguments:

function(a, b)  # provide argument 1 and 2

function(a) # only provide argument 1

function(b) still argument 1, not argument 2

[...]
> The problem is if you want a non-key word to be used as a default and 
> then also want to gather up additional positional arguments and so on. 
> There is no way designed to do that and possibly there should not be.

I'm open to suggestions for good designs that allow more flexible 
calling conventions, but as Python exists now, you can't skip over 
positional arguments except at the end.

> So what I am trying to understand is this. When someone types in a 
> function invocation and it is evaluated, when does the programmer get 
> a chance to intervene?

Aside from compile-time errors, any exception can be caught and 
processed *by the caller*:

try:
function(a, b)
except TypeError as e:
...

Python also allows you to customize the way exceptions are printed:

https://docs.python.org/3/library/sys.html#sys.excepthook

But the callee -- in this case, "function()" -- doesn't get a chance to 
customize the TypeError generated by the interpreter when you pass the 
wrong number of arguments, or an invalid keyword argument.

(Aside: it is unfortunate that TypeError gets used for both actual type 
errors, and mismatched arguments/parameter errors.)

[...]
> Is there a way to create a function and set it up so you have some 
> control over the error message?

Not easily.

You could, I suppose, take over parameter parsing yourself. Write your 
function like this:

def 

Re: [Tutor] Defining variable arguments in a function in python

[Steven D'Aprano]

On Sun, Dec 30, 2018 at 12:07:20AM -0500, Avi Gross wrote:

[...]
> Or on a more practical level, say a function wants an input from 1 to 10.
> The if statement above can be something like:
> 
> >>> def hello(a, *n, **m) :
>   if not (1 <= a <= 10) : a=5
>   print(a)
>   print(*n)
> 
>   
> >>> hello(1,2,3)
> 1
> 2 3
> >>> hello(21,2,3)
> 5
> 2 3
> >>> hello(-5,2,3)
> 5
> 2 3


This design is an example of "an attractive nuisance", possibly even a 
"bug magnet". At first glance, when used for mickey-mouse toy examples 
like this, it seems quite reasonable:

hello(999, 1, 2)  # I want the default value instead of 999

but thinking about it a bit more deeply, and you will recognise some 
problems with it.

First problem: 

How do you know what value to pass if you want the default? Is 999 out 
of range? How about 11? 10? Who knows? If you have to look up the docs 
to know what counts as out of range, you might as well read the docs to 
find out what the default it, and just pass that:

hello(5, 1, 2)  # I want the default value 5

but that kind of defeats the purpose of a default. The whole point of a 
default is that you shouldn't need to pass *anything at all*, not even a 
placeholder.

(If you need a placeholder, then you probably need to change your 
function parameters.)

But at least with sentinels like None, or Ellipsis, it is *obvious* 
that the value is probably a placeholder. With a placeholder like 11 or 
999, it isn't. They look like ordinary values.


Second problem:

Most of the time, we don't pass literal values to toy functions. We do 
something like this example:

for number, widget_ID, delivery_date in active_orders:
submit_order(number, widget_ID, delivery_date)

Can you see the bug? Of course you can't. There's no obvious bug. But 
little do you know, one of the orders was accidentally entered with an 
out-of-range value, let's say -1, and instead of getting an nice error 
message telling you that there's a problem that you need to fix, the 
submit_order() function silently replaces the erroneous value with the 
default.

The bug here is that submit_order() function exceeds its authority.

The name tells us that it submits orders, but it also silently decides 
to change invalid orders to valid orders using some default value. But 
this fact isn't obvious from either the name or the code. You only learn 
this fact by digging into the source code, or reading the documentation, 
and let's be honest, nobody wants to do either of those unless you 
really have to.

So when faced with an invalid order, instead of getting an error that 
you can fix, or even silently skipping the bad order, the submit_order() 
function silently changes it to a valid-looking but WRONG order that you 
probably didn't want. And that costs real money.

The risk of this sort of bug comes directly from the design of the 
function. While I suppose I must acknowledge that (hypothetically) 
there could be use-cases for this sort of design, I maintain that in 
general this design is a bug magnet: responsibility for changing 
out-of-range values to in-range values belongs with the caller, not 
the called function.

The caller may delegate that responsibility to another:

for number, widget_ID, delivery_date in active_orders:
number = validate_or_replace(number)
submit_order(number, widget_ID, delivery_date)

which is fine because it is explicit and right there in plain sight.

This then allows us to make the submit_order() far more resiliant: if it 
is passed an invalid order, it can either fail fast, giving an obvious 
error, or at least skip the invalid order and notify the responsible 
people.


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Defining variable arguments in a function in python

[Steven D'Aprano]

On Sat, Dec 29, 2018 at 11:42:16AM +0530, Karthik Bhat wrote:
> Hello,
> 
> I have the following piece of code. In this, I wanted to make use
> of the optional parameter given to 'a', i.e- '5', and not '1'
> 
> def fun_varargs(a=5, *numbers, **dict):
[...]
> 
> fun_varargs(1,2,3,4,5,6,7,8,9,10,Jack=111,John=222,Jimmy=333)
> 
> How do I make the tuple 'number' contain the first element to be 1 and not 2?


You can't. Python allocates positional arguments like "a" first, and 
only then collects whatever is left over in *numbers. How else would you 
expect it to work? Suppose you called:

fun_varargs(1, 2, 3)

wanting a to get the value 1, and numbers to get the values (2, 3). And 
then immediately after that you call 

fun_varargs(1, 2, 3)

wanting a to get the default value 5 and numbers to get the values 
(1, 2, 3). How is the interpreter supposed to guess which one you 
wanted?

If you can think of a way to resolve the question of when to give "a" 
the default value, then we can help you program it yourself:


def func(*args, **kwargs):
if condition:
# When?
a = args[0]
numbers = args[1:]
else:
a = 5  # Default.
numbers = args
...

But writing that test "condition" is the hard part.




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Fri, Dec 28, 2018 at 10:39:53PM -0500, Avi Gross wrote:
> I will answer this question then head off on vacation.

You wrote about 140 or more lines, but didn't come close to answering 
the question: how to randomly split data from a dictionary into training 
data and reserved data.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Fri, Dec 28, 2018 at 03:34:19PM -0500, Avi Gross wrote:

[...]
> You replied to one of my points with this about a way to partition data:
> 
> ---
> The obvious solution:
> 
> keys = list(mydict.keys())
> random.shuffle(keys)
> index = len(keys)*3//4
> training_data = keys[:index]
> reserved = keys[index:]
> ---
> 
> (In the above, "---" is not python but a separator!)
> 
> That is indeed a very reasonable way to segment the data. But it sort of
> makes my point. If the data is stored in a dictionary, the way to access it
> ended up being to make a list and play with that. I would still need to get
> the values one at a time from the dictionary such as in the ways you also
> show and I omit.

Yes? How else do you expect to get the value given a key except by 
looking it up?


> For me, it seems more natural in this case to simply have the data in 
> a data frame where I have lots of tools and methods available.


I'm not sure if your understanding of a data frame is the same as my
understanding. Are you talking about this?

http://www.r-tutor.com/r-introduction/data-frame

In other words, a two-dimensional array of some sort?

Okay, you have your data frame. Now what? How do you solve the problem
being asked? I'm not interested in vague handwaving that doesn't solve
anything. You specified data in a key:value store, let's say like this:


mydict = {'spam': 25, 'ham': 2, 'eggs': 7, 'cheddar': 1, 'brie': 14,
  'aardvark': 3, 'argument': 11, 'parrot': 16}

Here it is as a data frame:

df = [['spam', 'ham', 'eggs', 'cheddar', 'brie', 'aardvark', 'argument', 
'parrot'],
  [25, 2, 7, 1, 14, 3, 11, 16]]

Now what? How do you randomly split that into randomly selected set of
training data and reserved data?

Feel free to give an answer in terms of R, provided you also give an
answer in terms of Python. Remember that unlike R, Python doesn't have a
standard data frame type, so you are responsible for building whatever
methods you need.




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interpreter pasting Question

[Steven D'Aprano]

On Fri, Dec 28, 2018 at 12:58:00AM -0500, Avi Gross wrote:

[...]
> Copying and pasting multiple lines into the interpreter fails in mysterious
> ways, unless they are a logical single entity.
> 
> Is there a way to change this behavior, or perhaps an editor/environment
> that feeds multiple lines more carefully to the interpreter?

Which interpreter are you using?

If it is the regular Python REPL (Read Eval Print Loop), then pasting 
multiple lines should work, with some limitations. For example, I can 
paste:

x = 5
y = 6

as two lines, and it works fine under Linux. I see no reason why it 
should be different under Windows.

If you just run "python" in the Windows shell (cmd.exe or whatever its 
called), you should get an interactive interpreter. What happens when 
you paste multiple lines in that?



[...]
> When copied in looks like this:
> 
> >>> X=5
> 
> Y=6
> 
> SyntaxError: multiple statements found while compiling a single statement

That looks like a bug. Are you using IDLE? Perhaps it has been fixed 
in newer versions of Python, and if not, you should report it as a 
bug on the bugtracker

https://bugs.python.org/

but:

(1) try searching for similar bug reports first;
(2) read this first: 

http://www.sscce.org/

(3) and do try to keep your bug report short and to the point.


It looks like others have this problem with IDLE too:

https://duckduckgo.com/?q=idle+paste+multiple+lines


IPython/Jupyter allows pasting of multiple lines; I expect that bpython 
will too.

https://ipython.org/

https://bpython-interpreter.org/


But as I said, the vanilla Python REPL ought to work. How are you 
starting the interpreter? My guess is that you're using IDLE.



[...]
> Python has an eval() and an exec() and I would assume the latter would be a
> way to see what works.

No, exec() executes Python code, it doesn't try to simulate a REPL.


> Here are three lines using \n:
> 
> >>> exec("x=6\ny=7\nprint(x+y)")
> 13
>
> 
> That seems to work. Again, if I copied and pasted the same as three lines,
> it fails.

Without knowing the system you are using, and how you copy and paste, it 
is hard to comment except to say "Works for me". Here are three lines:

x = 6
y = 7
print(x + y)


If I select those three lines with the mouse, copy, then paste into a 
standard Python interactive interpreter, I get this:


py> x = 6
py> y = 7
py> print(x + y)
13


exactly as expected. But if I do it in IDLE, I get this:

>>> x = 6
y = 7
print(x + y)

SyntaxError: multiple statements found while compiling a single 
statement
>>> 

That *really* sounds like a bug to me. But perhaps I just don't 
understand IDLE.




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] dangerous class neighborhood

[Steven D'Aprano]

On Thu, Dec 27, 2018 at 09:48:02PM -0500, Avi Gross wrote:
> Sometimes when I post something I get back comments and evaluate them and
> learn quite a bit. I then reply and debate every little point and it can
> continue for a few rounds.

I think you sent this to the wrong mailing list. The original post and 
discussion was on Python-List, but you replied to Tutor.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Re Module

[Steven D'Aprano]

On Thu, Dec 27, 2018 at 08:40:12PM +0530, Asad wrote:
> Hi All ,
> 
>   I trying find a solution for my script , I have two files :
> 
> file1 - I need a search a error say x if the error matches
> 
> Look for the same error x in other file 2
> 
> Here is the code :
> I have 10 different patterns therefore I used list comprehension and
> compiling the pattern so I loop over and find the exact pattern matching
> 
> re_comp1 = [re.compile(pattern) for pattern in str1]


You can move the IGNORECASE flag into the call to compile. Also, perhaps 
you can use better names instead of "str1" (one string?).

patterns = [re.compile(pattern, re.IGNORECASE) for pattern in string_patterns]
 
> for pat in re_comp1:
> if pat.search(st,re.IGNORECASE):
> x = pat.pattern
> print x===> here it gives the expected output it correct
> match
> print type(x)
> 

Be careful here: even though you have ten different patterns, only *one* 
will be stored in x. If three patterns match, x will only get the last 
of the three and the others will be ignored.

 
> if re.search('x', line, re.IGNORECASE) is not None:  ===> Gives a wrong match

That's because you are trying to match the literal string "x", so it 
will match anything with the letter "x":

box, text, ax, equinox, except, hexadecimal, fix, Kleenex, sixteen ...


> Instead if I use :
> 
> if re.search(x, line, re.IGNORECASE) is not None: then no match occurs
>   print line

Here you are trying to match the variable called x. That is a very bad 
name for a variable (what does "x" mean?) but it should work.

If no match occurs, it probably means that the value of x doesn't occur 
in the line you are looking at.

Try printing x and line and see if they are what you expect them to be:

print x
print line


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Wed, Dec 26, 2018 at 11:02:07AM -0500, Avi Gross wrote:

> I often find that I try to make a main point ad people then focus on
> something else, like an example.

I can't speak for others, but for me, that could be because of a number 
of reasons:

- I agree with what you say, but don't feel like adding "I agree" 
after each paragraph of yours;

- I disagree, but can't be bothered arguing;

- I don't understand the point you intend to make, so just move on.

But when you make an obvious error, I tend to respond. This is supposed 
to be a list for teaching people to use Python better, after all.


> So, do we agree on the main point that choosing a specific data structure or
> algorithm (or even computer language) too soon can lead to problems that can
> be avoided if we first map out the problem and understand it better?

Sure, why not? That's vague and generic enough that it has to be true.

But if its meant as advice, you don't really offer anything concrete. 
How does one decide what is "too soon"? How does one avoid design 
paralysis?


> I do not concede that efficiency can be ignored because computers are fast.

That's good, but I'm not sure why you think it is relevant as I never 
suggested that efficiency can be ignored. Only that what people *guess* 
is "lots of data" and what actually *is* lots of data may not be the 
same thing.


> I do concede that it is often not worth the effort or that you can
> inadvertently make things worse and there are tradeoffs.

Okay.


> Let me be specific. The side topic was asking how to get a random key from
> an existing dictionary. If you do this ONCE, it may be no big deal to make a
> list of all keys, index it by a random number, and move on. I did supply a
> solution that might(or might not) run faster by using a generator to get one
> item at a time and stopping when found. Less space but not sure if less
> time.

Why don't you try it and find out?


> But what I often need to do is to segment lots of data into two piles. One
> is for training purposes using some machine learning algorithm and the
> remainder is to be used for verifications. The choice must be random or the
> entire project may become meaningless. So if your data structure was a
> dictionary with key names promptly abandoned, you cannot just call pop()
> umpteen times to get supposedly random results as they may come in a very
> specific order.

Fortunately I never suggested doing that.


> If you want to have 75% of the data in the training section,
> and 25% reserved, and you have millions of records, what is a good way to
> go? 

The obvious solution:

keys = list(mydict.keys())
random.shuffle(keys)
index = len(keys)*3//4
training_data = keys[:index]
reserved = keys[index:]

Now you have the keys split into training data and reserved data. To 
extract the value, you can just call mydict[some_key]. If you prefer, 
you can generate two distinct dicts:

training_data = {key: mydict[key] for key in training_data}

and similarly for the reserved data, and then mydict becomes redundant 
and you are free to delete it (or just ignore it).

Anything more complex than this solution should not even be attempted 
until you have tried the simple, obvious solution and discovered that it 
isn't satisfactory.

Keep it simple. Try the simplest thing that works first, and don't add 
complexity until you know that you need it.

By the way, your comments would be more credible if you had actual 
working code that demonstrates your point, rather than making vague 
comments that something "may" be faster. Sure, anything "may" be faster. 
We can say that about literally anything. Walking to Alaska from the 
southernmost tip of Chile while dragging a grand piano behind you "may" 
be faster than flying, but probably isn't. Unless you have actual code 
backing up your assertions, they're pretty meaningless.

And the advantage of working code is that people might actually learn 
some Python too.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Thu, Dec 27, 2018 at 07:03:18PM +, Mark Lawrence wrote:
> On 26/12/2018 00:00, Avi Gross wrote:
> >[Long enough that some should neither read nor comment on.]
> >
> 
> PLEASE GO AWAY YOU ARE REALLY IRRITATING.

People in glass houses...

Mark, you're not the arbiter of who is allowed to post here. You are 
being obnoxious. Please settle down and perhaps chill a bit. If you 
don't want to read Avi's posts, you know how to hit delete in your mail 
reader don't you?


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Tue, Dec 25, 2018 at 11:56:21PM -0500, Avi Gross wrote:

> I find that many people are fairly uncomfortable with abstraction and 
> tend to resist a pure top down approach by diving to any solutions 
> they may envision.

https://blog.codinghorror.com/it-came-from-planet-architecture/

> As someone asked on another python list, 
> is there a better way to get a random key for a dictionary. Well, not 
> easily without expanding all keys into a list of perhaps huge length. 

Define "better".

What do you value? Time, space, simplicity or something else?

One of the most harmful things to value is "cleverness" for its own 
sake. Some people tend to value a "clever" solution even when it wastes 
time, space and is over complex and therefore hard to maintain or debug.

Even when the practical response to the "clever" solution is "YAGNI".

What counts as "huge"? To me, picking a random key from a list of 100 
keys is "huge". Copy out 100 keys to a list by hand and then pick one? 
What a PITA that would be.

But to your computer, chances are that ten million keys is "small". One 
hundred million might be pushing "largish". A billion, or perhaps ten 
billion, could be "large". Fifty, a hundred, maybe even a thousand 
billion (a trillion) would be "huge".

Unless you expect to be handling at least a billion keys, there's 
probably no justification for anything more complex than:

random.choose(list(dict.keys())

Chances are that it will be faster *and* use less memory than any clever 
solution you come up with -- and even if it does use more memory, it 
uses it for a few milliseconds, only when needed, unlike a more complex 
solution that inflates the size of the data structure all the time, 
whether you need it or not.

Of course there may be use-cases where we really do need a more complex, 
clever solution, and are willing to trade off space for time (or 
sometimes time for space). But chances are YAGNI.


> Followed by a search of much of that list to get the nth index.

That's incorrect. Despite the name, Python lists aren't linked lists[1] 
where you have to traverse N items to get to the Nth item. They're 
arrays, where indexing requires constant time.


[...]
> If they keep demanding one function to master all, you can end up with 
> fairly awful spaghetti code.

https://en.wikipedia.org/wiki/God_object




[1] Technically speaking, this is not a requirement of the language, 
only a "quality of implementation" question. A Python interpreter could 
offer built-in lists using linked lists under the hood, with O(N) 
indexing. But all the major implementations -- CPython, Stackless, PyPy, 
Jython, IronPython, Cython, Nuitka, even (I think) MicroPython -- use 
arrays as the list implementation. Given how simple arrays are, I think 
it is fair to assume that any reasonable Python interpreter will do the 
same.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Tue, Dec 25, 2018 at 10:25:50PM -0500, Avi Gross wrote:

> class chainable_list(list):
> """Same as list but sort() can now be chained"""
> def chainsort(this, *args, **kwargs):
> this.sort(*args, **kwargs)
> return this

In Python, it is traditional to use "self" rather than "this" as the 
instance parameter.

Using "this" is not an error, but you can expect a lot of strange looks. 
Like a Scotsman in a kilt wandering down the middle of Main Street, 
Pleasantville USA.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] decomposing a problem

[Steven D'Aprano]

On Wed, Dec 26, 2018 at 01:06:04AM +, Alan Gauld via Tutor wrote:

> In Smalltalk the default return value from
> any method is self. In Python it is None.
> 
> self allows chaining of methods, None does not.


You might be interested in this simple recipe for retrofitting method 
chaining onto any class:

http://code.activestate.com/recipes/578770-method-chaining-or-cascading/


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] loop error

[Steven D'Aprano]

On Thu, Dec 20, 2018 at 10:47:44PM +0100, Aine Gormley wrote:
> Hello, could somebody take a quick look at my code? I am unsure why I am
> getting a loop error?

That's hard to do if you don't show us the code :-) 

Please COPY AND PASTE (don't try to retype it from memory) the MINIMUM 
amount of code which actually runs. If you're unsure about cutting the 
code down to the minimum that demonstrates the error, please feel free 
to ask. You can also read this:

http://www.sscce.org/


Its written for Java programmers, but applies to any programming 
language including Python.

And what is "a loop error"? Please COPY AND PASTE the full exception. It 
should start with a line:

Traceback...

and end with some sort of error message.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python

[Steven D'Aprano]

On Thu, Dec 20, 2018 at 10:49:25AM -0500, Mary Sauerland wrote:

> I want to get rid of words that are less than three characters


> f1_name = "/Users/marysauerland/Documents/file1.txt"
> #the opinions
> f2_name = "/Users/marysauerland/Documents/file2.txt"
> #the constitution

Better than comments are meaningful file names:

opinions_filename = "/Users/marysauerland/Documents/file1.txt"
constitution_filename = "/Users/marysauerland/Documents/file2.txt"

> def read_words(words_file):
> return [word.upper() for line in open(words_file, 'r') for word in 
> line.split()]

Don't try to do too much in a single line of code. While technically 
that should work (I haven't tried it to see that it actually does) it 
would be better written as:

def read_words(words_file):
with open(words_file, 'r') as f:
return [word.upper() for line in f for word in line.split()]

This also has the advantage of ensuring that the file is closed after 
the words are read. In your earlier version, it is possible for the file 
to remain locked in an open state.

Note that in this case Python's definition of "word" may not agree with 
the human reader's definition of a word. For example, Python, being 
rather simple-minded, will include punctuation in words so that 

"HELLO"
"HELLO."

count as different words. Oh well, that's something that can be adjusted 
later. For now, let's just go with the simple-minded definition of a 
word, and worry about adjusting it to something more specialised later.



> read_words(f1_name)
> #performs the function on the file

The above line of code (and comment) are pointless. The function is 
called, the file is read, the words are generated, and then immediately 
thrown away. To use the words, you need to assign them to a variable, as 
you do below:


> set1 = set(read_words(f1_name))
> #makes each word into a set and removes duplicate words

A meaningful name is better. Also the comment is inaccurate: it is not 
that *each individual* word is turned into a set, but that the *list* of 
all the words are turned into a set. So better would be:

opinions_words = set(read_words(opinions_filename))
constitition_words = set(read_words(constitution_filename))

This gives us the perfect opportunity to skip short words:

opinions_words = set(
word for word in read_words(opinions_filename) if len(word) >= 3)
constitition_words = set(
word for word in read_words(constitution_filename) if len(word) >= 3)


Now you have two sets of unique words, each word guaranteed to be at 
least 3 characters long.

The next thing you try to do is count how many words appear in each set. 
You do it with a double loop:

> count_same_words = 0
> for word in set1:
> if word in set2:
> count_same_words += 1

but the brilliant thing about sets is that they already know how to do 
this themselves! Let's see the sorts of operations sets understand:


py> set1 = set("abcdefgh")
py> set2 = set("defghijk")
py> set1 & set2  # the intersection (overlap) of both sets
{'h', 'd', 'f', 'g', 'e'}
py> set1 | set2  # the union (combination) of both sets
{'f', 'd', 'c', 'b', 'h', 'i', 'k', 'j', 'a', 'g', 'e'}
py> set1 ^ set2  # items in one or the other but not both sets
{'i', 'k', 'c', 'b', 'j', 'a'}
py> set1 - set2  # items in set1 but not set2
{'c', 'b', 'a'}


(In the above, "py>" is the Python prompt. On your computer, your prompt 
is probably set to ">>>".)

Can you see which set operation, one of & | ^ or - , you would use to 
get the set of words which appear in both sets? Hint: it isn't the - 
operation. If you wanted to know how many words appear in the 
constitution but NOT in the opinions, you could write:


word_count = len(constitition_words - opinions_words)


Does that give you a hint how to approach this?



Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Obfuscated Python [was Long Lines techniques]

[Steven D'Aprano]

On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote:

> Python may claim to be straightforward but I can easily see ways
> to fool people in python too with dunder methods or function closures or
> decorators or ...

Dunder methods shouldn't fool anyone. Each dunder method has a 
straightforward use, and most of the time you can guess that they do. 
__add__ implements the + operator, etc.

Closures and decorators can be abused, as can anything. One can write 
obfuscated code in any language. Here's some amusing obfuscated Python:

https://www.reddit.com/r/Python/comments/i1qgp/reduce_and_lambda_two_great_tastes_that_taste/

https://mail.python.org/pipermail/python-list/2013-October/658470.html

Anyone else want to share some obfuscated Python code?



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Steven D'Aprano]

On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote:

[...]
> There are cases where it may make sense to have a long like connected by AND
> or OR given how python does short-circuiting while returning the last thing
> or two it touched instead of an actual True/False. For example, you may want
> to take the first available queue that is not empty with something like
> this:
> 
> Using = A or B or C or ... or Z
> Handling = Using.pop()
> 
> Sure, that could be rewritten into multiple lines. 

using = (A or B
 or C or D 
 or E or F)


[...]
> I recently wrote some code and ran into error messages on lines I was trying
> to keep short:
> 
> A = 'text"
> A += "more text"
> A += object
> A+= ...

Without knowing the error message, its impossible to say what the 
problem is. My guess is that the object on line three wasn't a string. 
In which case, the obvious fix is:

A += str(object)



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Steven D'Aprano]

On Fri, Dec 14, 2018 at 01:03:55AM +, Alan Gauld via Tutor wrote:

> I'd probably suggest
> 
> stgs = ''.join([
> "long string",
> "another string",
> ...
> "last string"
> ])

That's certainly better than using the + operator, as that will be quite 
inefficient for large numbers of strings. But it still does unnecessary 
work at runtime that could be done at compile time.

Hence my preferred solution is implicit string concatenation:

stgs = ("long string"
"another string"
"last string")

Note the lack of commas.


[...]
> > or even generator expression. 
> 
> These are usually just parens in a function. The generator
> expression is the bit inside (not including) the parens.

Hmmm, well, yes, no, maybe. The definition of a generator comprehension 
*requires* it to be surrounded by parentheses. You cannot write this:

gen = x + 1 for x in items

you must use brackets of some form:

- [] for a list comprehension;
- () for a generator comprehension;
- {} for a dict or set comprehension;

But for the generator case, a short-cut is allowed. If the expression is 
already surrounded by parens, as in a one-argument function call, you 
can forego the inner brackets. So instead of:

flag = any((condition(x) for x in items))

we can just write:

flag = any(condition(x) for x in items)



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Long Lines techniques

[Steven D'Aprano]

On Thu, Dec 13, 2018 at 12:36:27PM -0500, Avi Gross wrote:

> Simple question:
> 
> When lines get long, what points does splitting them make sense and what
> methods are preferred?

Good question!

First, some background:

Long lines are a potential code smell: a possible sign of excessively 
terse code. A long line may be a sign that you're doing too much in one 
line.

https://martinfowler.com/bliki/CodeSmell.html
http://wiki.c2.com/?CodeSmell
https://blog.codinghorror.com/code-smells/

Related: 
https://www.joelonsoftware.com/2005/05/11/making-wrong-code-look-wrong/

Note that merely splitting a logical line over two or more physical 
lines may still be a code-smell. Sure, your eyes don't get as tired 
reading fifteen lines of 50 characters each, compared to a single 750 
character line, but there's just as much processing going on in what is 
essentially a single operation.

Long lines are harder to read: your eyes have to scan across a long 
line, and beyond 60 or 70 characters, it becomes physically more 
difficult to scan across the line, and the error rate increases. 
[Citation required.]

But short lines don't include enough information, so the traditional 
compromise is 80 characters, the character width of the old-school 
green-screen terminals. The Python standard library uses 79 characters. 
(The odd number is to allow for scripts which count the newline at the 
end of the line as one of the 80.)

https://www.python.org/dev/peps/pep-0008/


Okay, so we have a style-guide that sets a maximum line length, whether 
it is 72 or 79 or 90 or 100 characters. What do you do when a line 
exceeds that length?

The only firm rule is that you must treat each case on its own merits. 
There is no one right or wrong answer. Every long line of code is 
different, and the solution will depend on the line itself. There is no 
getting away from human judgement.


(1) Long names. Do you really need to call the variable 
"number_of_characters" when "numchars" or even "n" would do?

The same applies to long function names: "get_data_from_database" is 
probably redundant, "get_data" will probably do.

Especially watch out for long dotted names that you use over and over 
again. Unlike static languages like Java, each dot represents a runtime 
lookup. Long names like:

package.subpackage.module.object.method

requires four lookups. Look for oportunities to make an alias for a long 
name and avoid long chains of dots:

for item in sequence:
do_something_with(package.subpackage.module.object.method(arg, item))

can be refactored to:

method = package.subpackage.module.object.method
for item in sequence:
do_something_with(method(arg, item))

and is both easier to read and more efficient. A double win!


(2) Temporary constants: sometimes it is good enough to just introduce a 
simple named constant used once. The cognitive load is low if it is 
defined immediately before it is used. Instead of the long line:

raise ValueError("expected a list, string, dict or None, but instead got 
'%s'" % type(value).__name__)

I write:

errmsg = "expected a list, string, dict or None, but instead got '%s'"
raise ValueError(errmsg % type(value).__name__)


(3) Code refactoring. Maybe that long line is sign that you need to add 
a method or function? Especially if you are using that line, or similar, 
in multiple places. But refactoring is justified even if you use the 
line *once* if it is complicated enough.

Likewise, sometimes it is helpful to factor out separate sub-expressions 
onto their own lines, using their own variables, rather than doing 
everything in a single, complicated, expression.

Psychologists, educators and linguists call this "chunking", and it is 
often very helpful for simplifying complicated ideas, sentences and 
expressions.

The lack of chunks is why long Perl one-liners are so inpenetrable.


(4) Split the long logical line over multiple physical lines. This does 
nothing to reduce the inherent complexity of the line, but if that's 
fairly low to start with, it is often helpful.

Python gives us two ways to split a logical line over multiple physical 
lines: a backslash at the end of the line, and brackets of any sort.

The preferred way is to use round brackets for grouping:

result = (some very long expression
  which can be split over
  many lines)

This is especially useful with function calls:

result = function(first_argument, second_argument,
  third_argument, fourth_argument)

If you are building a list or dict literal, there is no need for the 
parentheses, as square and curly brackets have the same effect. That's 
especially useful with two-dimensional nested lists:

data = [[row, one, with, many, items],
[row, two, with, many, items],
[row, three, with, many, items]]

For long strings, I like to use *implicit string concatentation*. String 
literals which are separated by 

Re: [Tutor] Python script errors

[Steven D'Aprano]

On Wed, Dec 12, 2018 at 06:57:09AM -0600, Ravi Kumar wrote:

> I know this occurs when the api response is nulls but how do I handle this?

if response is None:
handle None case
else:
handle non-None case



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python_About "Guess my number" program

[Steven D'Aprano]

Hi Dat, and welcome!

On Wed, Dec 12, 2018 at 11:26:30AM +0600, Hoang Quoc Dat wrote:
[...]
> After a while, I went on and search on the internet and
> find out 1 similar program but the coding logic is like: The computer keeps
> guessing the middle number of the range and narrow down the range by
> focusing on only the upper/lower part of the range after receving our
> answer.

Yes, that is the classic algorithm. It is a variant of binary search.


> I somehow have come to believe that the Python language and our logical
> thinking can do better than that option of choosing the middle number all
> the time but I still cannot find out the proper answer (probably because I
> am very much new to this). Could someone provide answer for this program?

If the only information you have is whether the secret number is 
"higher" or "lower", then no, there is no better algorithm.

If you had more information, like "red hot, hot, warmer, warm, cool, 
cold, colder, ice cold" to indicate how close the guess is, then you 
might be able to do better, if you had some objective rule for deciding 
how close "warm" etc was.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Steven D'Aprano]

On Wed, Dec 12, 2018 at 12:52:09AM -0500, Avi Gross wrote:
> Asad,
> 
> I wonder if an import from __future__ happened, perhaps in the version of
> collections you used. Later versions of 2.x allow optional use of the 3.x
> style of print.


The effect of __future__ imports, like any other import, is only within 
the module that actually does the import. Even in the unlikely event 
that collections did such a future import, it would only effect print 
within that module, not globally or in the interactive interpreter.

Here's a demo:

# prfunc_demo.py
from __future__ import print_function

try:
exec("print 123")
except SyntaxError:
print("old style print failed, as expected")
print("as print is now a function")



And importing it into the interactive interpreter shows that the effect 
of the future import is localised:

[steve@ando ~]$ python2.6
Python 2.6.7 (r267:88850, Mar 10 2012, 12:32:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> import prfunc_demo
old style print failed, as expected
as print is now a function
py> print "But here in the REPL, nothing has changed."
But here in the REPL, nothing has changed.







-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Steven D'Aprano]

On Tue, Dec 11, 2018 at 09:07:58PM +0530, Asad wrote:
> Hi All,
> 
>   I used your solution , however found a strange issue with deque :

No you haven't. You found a *syntax error*, as the exception says:

> >>> print 'Deque:', d
>   File "", line 1
> print 'Deque:', d
>  ^
> SyntaxError: invalid syntax

which means the error occurs before the interpreter runs the code. You 
could replace the above line with any similar line:

print 'Not a deque', 1.2345

and you will get the same error.

When you are faced with an error in the interactive interpreter, you 
should try different things to see how they effect the problem. Does the 
problem go away if you use a float instead of a deque? If you change the 
string, does the problem go away? If you swap the order, does the 
problem go away? What if you use a single value instead of two?

This is called "debugging", and as a programmer, you need to learn how 
to do this.


[...]
> In python 2.6 print statement work as print "Solution"
> however after import collection I have to use print with print("Solution")
> is this a known issue ?

As Peter says, you must have run 

from __future__ import print_function

to see this behaviour. This has nothing to do with import collection. 
You can debug that for yourself by exiting the interactive interpreter, 
starting it up again, and trying to print before and after importing 
collection.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Steven D'Aprano]

On Sun, Dec 09, 2018 at 03:45:07PM +0530, Asad wrote:
> Hi All ,
> 
>   I have the following code to search for an error and prin the
> solution .

Please tidy your code before asking for help optimizing it. We're 
volunteers, not being paid to work on your problem, and your code is too 
hard to understand.

Some comments:


> f4 = open (r" /A/B/file1.log  ", 'r' )
> string2=f4.readlines()

You have a variable "f4". Where are f1, f2 and f3?

You have a variable "string2", which is a lie, because it is not a 
string, it is a list.

I will be very surprised if the file name you show is correct. It has a 
leading space, and two trailing spaces.


> for i in range(len(string2)):
> position=i

Poor style. In Python, you almost never need to write code that iterates 
over the indexes (this is not Pascal). You don't need the assignment 
position=i. Better:

for position, line in enumerate(lines):
...


> lastposition =position+1

Poorly named variable. You call it "last position", but it is actually 
the NEXT position.


> while True:
>  if re.search('Calling rdbms/admin',string2[lastposition]):

Unnecessary use of regex, which will be slow. Better:

if 'Calling rdbms/admin' in line:
break


>   break
>  elif lastposition==len(string2)-1:
>   break

If you iterate over the lines, you don't need to check for the end of 
the list yourself.


A better solution is to use the *accumulator* design pattern to collect 
a block of lines for further analysis:

# Untested.
with open(filename, 'r') as f:
block = []
inside_block = False
for line in f:
line = line.strip()
if inside_block:
if line == "End of block":
inside_block = False
process(block)
block = []  # Reset to collect the next block.
else:
block.append(line)
elif line == "Start of block":
inside_block = True
# At the end of the loop, we might have a partial block.
if block:
 process(block)


Your process() function takes a single argument, the list of lines which 
makes up the block you care about.

If you need to know the line numbers, it is easy to adapt:

for line in f:

becomes:

for linenumber, line in enumerate(f):
# The next line is not needed in Python 3.
linenumber += 1  # Adjust to start line numbers at 1 instead of 0

and:
 
block.append(line)

becomes 

block.append((linenumber, line))


If you re-write your code using this accumulator pattern, using ordinary 
substring matching and equality instead of regular expressions whenever 
possible, I expect you will see greatly improved performance (as well as 
being much, much easier to understand and maintain).



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Increase performance of the script

[Steven D'Aprano]

On Sun, Dec 09, 2018 at 03:45:07PM +0530, Asad wrote:
> Hi All ,
> 
>   I have the following code to search for an error and prin the
> solution .
> 
> /A/B/file1.log size may vary from 5MB -5 GB
[...]

> The problem I am facing in performance issue it takes some minutes to print
> out the solution . Please advice if there can be performance enhancements
> to this script .

How many minutes is "some"? If it takes 2 minutes to analyse a 5GB file, 
that's not bad performance. If it takes 2 minutes to analyse a 5MB file, 
that's not so good.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Weird Unicode encode/decode errors in Python 2

[Steven D'Aprano]

This is not a request for help, but a demonstration of what can go wrong 
with text processing in Python 2.

Following up on the "Special characters" thread, one of the design flaws 
of Python 2 is that byte strings and text strings offer BOTH decode and 
encode methods, even though only one is meaningful in each case.[1]

- text strings are ENCODED to bytes;
- byte are DECODED to text strings.

One of the symptoms of getting it wrong is when you take a Unicode text 
string and encode/decode it but get an error from the *opposite* 
operation:


py> u'ä'.decode('latin1')
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in 
position 0: ordinal not in range(128)


Look at what happens: I try to DECODE a string, but get an ENCODE error. 
And even though I specified Latin 1 as the codec, Python uses ASCII. 
What is going on here?

Behind the scenes, the interpreter takes my text u'ä' (a Unicode string) 
and attempts to *encode* it to bytes first, using the default ASCII 
codec. That fails. Had it succeeded, it would have then attempted to 
*decode* those bytes using Latin 1.

Similarly:

py> b = u'ä'.encode('latin1')
py> print repr(b)
'\xe4'
py> b.encode('latin1')
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0: 
ordinal not in range(128)


The error here is that I tried to encode a bunch of bytes, instead of 
decoding them. But the insidious thing about this error is if you are 
working with pure ASCII, it seems to work:

py> 'ascii'.encode('utf-16')
'\xff\xfea\x00s\x00c\x00i\x00i\x00'

That is, it *seems* to work because there's no error, but the result is 
pretty much meaningless: I *intended* to get a UTF-16 Unicode string, 
but instead I ended up with bytes just like I started with.

Python 3 fixes this bug magnet by removing the decode method from 
Unicode text strings, and the encode method from byte-strings.



[1] Technically this is not so, as there are codecs which can be used to 
convert bytes to bytes, or text to text. But the vast majority of common 
cases, codecs are used to convert bytes to text and vice versa. For the 
rare exception, we can use the "codecs" module.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Trouble in dealing with special characters.

[Steven D'Aprano]

On Sun, Dec 09, 2018 at 09:23:59AM +1100, Cameron Simpson wrote:
> On 07Dec2018 21:20, Steven D'Aprano  wrote:


# Python 2
> >>>>txt = "abcπ"
> >
> >but it is a lie, because what we get isn't the string we typed, but the
> >interpreters *bad guess* that we actually meant this:
> >
> >>>>txt
> >'abc\xcf\x80'
> 
> Wow. I did not know that! I imagined Python 2 would have simply rejected 
> such a string (out of range characters -- ordinals >= 256 -- in a "byte" 
> string).

Nope.

Python 2 tries hard to make bytes and unicode text work together. If 
your strings are pure ASCII, it "Just Works" and it seems great but on 
trickier cases it can lead to really confusing errors.

Behind the scenes, what the interpreter is doing is using some platform- 
specific codec (ASCII, UTF-8, or similar) to automatically encode/decode 
from bytes to text or vise versa. This sort of "Do What I Mean" 
processing can work, up to the point that it doesn't, then it all goes 
pearshaped and you have silent failures and hard-to-diagnose errors.

That's why Python 3 takes a hard-line policy that you cannot mix text 
and bytes (except, possibly, if one is the empty string) except by 
explicitly converting from one to the other.

-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Steven D'Aprano]

On Fri, Dec 07, 2018 at 12:30:18PM -0500, Avi Gross wrote:

> #  > I have often seen something like this done with methods, such as to
> #  > emulate decorator functionality where a method is created in an 
> #  > object with a name
> #  > and the very next method created has the same name with the same
> #  > function name as an argument to replace it.
> #  

Then I asked:

> # I don't understand this. Can you show an example? Even if made up?
[...]

And Avi explained:

> class classy(object):
> ...
> # Original function first of this name
> def recreational_fun(args):
> ...
> 
> # New function, second of this name
> recreational _fun = timing_is_everything(recreational _fun, args)

Ah, now I understand!

That's a decorator. The only difference is that it predates the 
introduction of *decorator syntax* using the @ symbol.

The *decorator design pattern* is an idiom which uses a wrapper function 
to add functionality to an existing function. Given a language where 
functions are first-class values, you can pass a function to another 
function as argument, and return yet a third newly constructed function. 
So this gives us the decorate pattern. In pseudo-code:

# Decorator pattern
def decorate(input):
output = wrap input with extra functionality
return output

new_function = decorate(function)

But if you don't need to keep both the old and new versions of the 
function, you can just write this:

function = decorate(function)


https://en.wikipedia.org/wiki/Decorator_pattern

In Java, it is more common to apply the decorator pattern to instances 
rather than functions or classes, but the principle is the same. Even if 
Java makes something conceptually simple incredibly complex :-)

*Technically* speaking, such a design could have been used in Python 
going all the way back to Python 1.4 or older, since functions have 
always been first-class values. But it only got really useful from about 
Python 2.2, with the introduction of closures to the language, and the 
classmethod and staticmethod decorators.

But initially, we didn't have dedicated syntax for applying a decorator, 
and had to write:

def function():
...

function = decorate(function)


which repeats the function name three times. It wasn't until 2.4 that 
the @ syntax was approved:

@decorate
def function():
...


which is literally syntactic sugar for the original version.

See PEP 318 for the background:

https://www.python.org/dev/peps/pep-0318/




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Steven D'Aprano]

On Fri, Dec 07, 2018 at 05:59:22PM +, Alan Gauld via Tutor wrote:

[...]
> > ... In languages without garbage collection, reusing
> > the same name, like "index" repeatedly might save some 
> > small amount of space. 
> 
> Garbage collection only helps if the variable loses
> its assignment. If the variable retains its value it
> won't be garbage collected. So the same space saving applies.
> 
> However it is very rare that Python is used in the kind
> of scenarios where that kind of space saving is
> significant (usually embedded controllers with
> tiny memory spaces).

People routinely use Python with seriously large amounts of data, 
hundreds of megabytes or even gigabytes, and freely make copies of it, 
secure in the knowledge that they don't have to do much to manage memory 
because the garbage collector will do so for them.

Consider a simple expression like:

text = '"' + (text.strip().lower().replace(',', ' ')) + '"'

That makes five slightly-modified copies of the original text. And 
that's just from a single short expression. Memory is cheap, but not so 
cheap that we can afford to keep around hundreds of redundant copies of 
large objects.

Generally speaking, we can afford to be profligate with copies of 
objects because we know they won't "leak" and survive for very long: at 
worst, they will be reclaimed when the current function returns. The 
only times we need worry about lifetimes of objects are when we are 
working in the global scope, or when adding attributes to long-lived 
objects. Regardless of whether we write the above as a single 
expression, or split it over many lines:

text = text.strip()
text = text.lower()
text = text.replace(',', ' ')
text = '"' + text
text = text + '"'

the result will be the same. And I don't think the above would be more 
understandable if we invented separate names for each of the 
intermediate results.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Steven D'Aprano]

On Thu, Dec 06, 2018 at 09:13:01PM -0500, Avi Gross wrote:

> But so much code I see in python does not only reuse the same variable names
> but in a confusing way.
> 
> file =  "some name"
> file = open(file, "r")
> file = some_wrapper(file)

I agree this is confusing: you have the same name, "file", used to mean 
a file name, an open file object, and whatever some_wrapper returns. 
This is not the clearest code I've ever seen.


> mylist = [ ... ]
> mylist = sorted(mylist)

There's nothing wrong with that, except that it ought to be written:

mylist = [ ... ]
mylist.sort()

to sort in place instead of making a sorted copy.



> for index in range(...):
>   stuff
> 
> for index in open(...)
>   more stuff


Using "index" to iterate over the lines in a file is just a crappy 
choice of name.



[...]
> I have often seen something like this done with methods, such as to emulate
> decorator functionality where a method is created in an object with a name
> and the very next method created has the same name with the same function
> name as an argument to replace it.

I don't understand this. Can you show an example? Even if made up?


> So is there a guide on when reuse is good and when it just obfuscates? What
> is good practice?

If you pick *meaningful* names, it will be pretty obvious when to reuse 
the same name: is the name still meaningful? Then you MAY reuse. If the 
name is not meaningful, then you MUST NOT reuse it, unless you are 
deliberately writing obfuscated code.

If you have a name "column" which you use for a columns, then you 
shouldn't reuse it for rows, or lines of text read from a file, or the 
length of a list, or the date. But you can reuse it for another column, 
provided there's no confusion over which column is which.

The same applies to any other language.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] playing sound files in python

[Steven D'Aprano]

On Fri, Dec 07, 2018 at 02:17:55AM +, nathan tech wrote:
> Hello all!
> 
> My name is nate, and I am relatively new to this list, relatively being 
> just signed up.
> 
> I have a question that you would think would be obvious, but alas I have 
> struggled to figure out.
> 
> How do I play sound files in python.

Alas, its not obvious nor easy.

https://duckduckgo.com/?q=python+play+sound+files

 
> More specificly, I want to play ogg files especially, with wav and mp3 
> also being a high priority.

I believe that wxPython supports .wav files, but not .ogg or .mp3.

Playing sound in Python is a definite weakness unless you have a 
third-party library like PyGame that supports it in a platform- 
independent way.

You could try:

print('\a')

but this requires that you are running in a terminal that supports the 
BEL character, that the system beep has not been turned off, and even if 
it works, its only a short beep.

If your OS (Windows?) has a sound file player, you could try 
calling out to it with os.system. Under Linux, I might try 
something like this:

os.system('mpg123 -q why_is_the_rum_gone-remix.mp3')

but that pauses until paying is over, and requires the mpg123 player. 
There are other players, like ogg123. What you get on Windows, I don't 
know.

Pygame is starting to sound more attractive :-)



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Trouble in dealing with special characters.

[Steven D'Aprano]

On Fri, Dec 07, 2018 at 02:06:16PM +0530, Sunil Tech wrote:
> Hi Alan,
> 
> I am using Python 2.7.8

That is important information.

Python 2 unfortunately predates Unicode, and when it was added some bad 
decisions were made. For example, we can write this in Python 2:

>>> txt = "abcπ"

but it is a lie, because what we get isn't the string we typed, but the 
interpreters *bad guess* that we actually meant this:

>>> txt
'abc\xcf\x80'

Depending on your operating system, sometimes you can work with these 
not-really-text strings for a long time, but when it fails, it fails 
HARD with confusing errors. Just as you have here:

> >>> tx = "MOUNTAIN VIEW WOMEN’S HEALTH CLINIC"
> >>> tx.decode()
> Traceback (most recent call last):
>   File "", line 1, in 
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 19:
> ordinal not in range(128)


Here, Python tried to guess an encoding, and picked some 
platform-specific encoding like Latin-1 or CP-1252 or something even 
more exotic. That is the wrong thing to do.

But if you can guess which encoding it uses, you can make it work:

tx.decode("Latin1")

tx.decode("CP-1252")

But a better fix is to use actual text, by putting a "u" prefix outside 
the quote marks:

txt = u"MOUNTAIN VIEW WOMEN’S HEALTH CLINIC"


If you need to write this to a file, you can do this:

file.write(txt.encode('utf-8'))

To read it back again:

# from a file using UTF-8
txt = file.read().decode('utf-8')

(If you get a decoding error, it means your text file wasn't actually 
UTF-8. Ask the supplier what it really is.)


> How to know whether in a given string(sentence) is there any that is not
> ASCII character and how to replace?

That's usually the wrong solution. That's like saying, "My program can't 
add numbers greater than 100. How do I tell if a number is greater than 
100, and turn it into a number smaller than 100?"

You can do this:

mystring = "something"
if any(ord(c) > 127 for c in mystring):
print "Contains non-ASCII"


But what you do then is hard to decide. Delete non-ASCII characters? 
Replace them with what?

If you are desperate, you can do this:

bytestring = "something"
text = bytestring.decode('ascii', errors='replace')
bytestring = text.encode('ascii', errors='replace')


but that will replace any non-ascii character with a question mark "?" 
which might not be what you want.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Trouble in dealing with special characters.

[Steven D'Aprano]

On Fri, Dec 07, 2018 at 01:28:18PM +0530, Sunil Tech wrote:
> Hi Tutor,
> 
> I have a trouble with dealing with special characters in Python 

There are no special characters in Python. There are only Unicode 
characters. All characters are Unicode, including those which are also 
ASCII.

Start here:

https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/

https://blog.codinghorror.com/there-aint-no-such-thing-as-plain-text/

https://www.youtube.com/watch?v=sgHbC6udIqc

https://nedbatchelder.com/text/unipain.html


https://docs.python.org/3/howto/unicode.html

https://docs.python.org/2/howto/unicode.html


Its less than a month away from 2019. It is sad and shameful to be 
forced to use only ASCII, and nearly always unnecessary. Writing code 
that only supports the 128 ASCII characters is like writing a calculator 
that only supports numbers from 1 to 10.

But if you really must do so, keep reading.


> Below is
> the sentence with a special character(apostrophe) "MOUNTAIN VIEW WOMEN’S
> HEALTH CLINIC" with actually should be "MOUNTAIN VIEW WOMEN'S HEALTH CLINIC
> ".

Actually, no, it should be precisely what it is: "WOMEN’S" is correct, 
since that is an apostrophe. Changing the ’ to an inch-mark ' is not 
correct. But if you absolutely MUST change it:

mystring = "MOUNTAIN VIEW WOMEN’S HEALTH CLINIC"
mystring = mystring.replace("’", "'")

will do it in Python 3. In Python 2 you have to write this instead:

# Python 2 only
mystring = u"MOUNTAIN VIEW WOMEN’S HEALTH CLINIC"
mystring = mystring.replace(u"’", u"'")


to ensure Python uses Unicode strings.

What version of Python are you using, and what are you doing that gives 
you trouble?

It is very unlikely that the only way to solve the problem is to throw 
away the precise meaning of the text you are dealing with by reducing it 
to ASCII.

In Python 3, you can also do this:

mystring = ascii(mystring)

but the result will probably not be what you want.


> Please help, how to identify these kinds of special characters and replace
> them with appropriate ASCII?

For 99.99% of the characters, there is NO appropriate ASCII. What 
ASCII character do you expect for these?

§ π Й খ ₪ ∀ ▶ 丕 ☃ ☺️

ASCII, even when it was invented in 1963, wasn't sufficient even for 
American English (no cent sign, no proper quotes, missing punctuation 
marks) let alone British English or international text.

Unless you are stuck communicating with an ancient program written in 
the 1970s or 80s that cannot be upgraded, there are few good reasons to 
cripple your program by only supporting ASCII text.

But if you really need to, this might help:

http://code.activestate.com/recipes/251871-latin1-to-ascii-the-unicode-hammer/

http://code.activestate.com/recipes/578243-repair-common-unicode-mistakes-after-theyve-been-m/




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Regarding Python api script

[Steven D'Aprano]

On Thu, Dec 06, 2018 at 08:17:23AM -0600, Ravi Kumar wrote:

> I know I am asking a lot 

Yes you are. Please read this:

http://sscce.org/


It is written for Java programmers, but it applies equally to all 
languages, including Python.

Think about how difficult a job you are giving us: we don't have access 
to your network; we don't have the API key (which is understandable); we 
can't run the code; we can't even be confident that the code you give us 
is the same code you are running.

In fact we know that it cannot be the same code, because what you have 
given us has a Syntax Error in line 8:

> organization='4

(missing quote mark). Where there is one change to the code, there could 
be dozens for all we know.


> but I hit a road block on this i am assuming One
> small change in my last for loop I can get this working for all devices

That doesn't seem like a reasonable assumption to me. You might be 
right, or it could take many large changes.


> and
> I have hardcoded for one network how do I use the same code for different
> networks as well

You can read the network from the command line, or a config file, or an 
environment variable, or interactively by asking the user to enter the 
network using the "input" (Python 3) or "raw_input" (Python 2) 
functions.

I expect that you need more than one piece of information to change the 
network, it looks like you probably need at least four:

serveraddress
merakiAPIkey
organization
networkid

so it probably is best to read the details from a config file:

https://docs.python.org/3/library/configparser.html

is probably simplest, but you could use Property Lists as well:

https://docs.python.org/3/library/plistlib.html




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Borrowing restricted code

[Steven D'Aprano]

On Thu, Dec 06, 2018 at 10:41:37AM +, Alan Gauld via Tutor wrote:
> On 06/12/2018 00:45, Steven D'Aprano wrote:
>
> > As for what is "not worth prosecuting", there are no copyright police 
> > who troll the internet looking for copied lines of code. Nobody is going 
> > to be scanning your Github repos looking for infringement (at least not 
> 
> Sorry, that's not strictly true. I know of at least two large companies
> who have full time teams whose job is to trawl Github, sourceforge
> and a few others looking at new checkins for unlicensed use of
> corporate code. And one of those teams is not even a technical
> team, they are corporate lawyers... And they do prosecute (or at least
> threaten to).

Ah, that's my error. When I said "copyright police", I was actually 
talking about *actual* law enforcement, not private companies. Sorry for 
not being more clear.


> > But they're not going to open up your Python folder and demand to see 
> > licences for everything or question whether or not you copy code from 
> > Stackoverflow without permission.
> 
> Again bodies like FAST(*) certainly do that (with police cooperation
> of course - they need a search warrant).

Again, I was specifically talking about customs agents.

I don't know who FAST is, but if they're like the Business Software 
Alliance they're effectively a front for Microsoft to strong-arm 
companies into buying Microsoft software under the threat of copyright 
infringement lawsuits.

https://en.wikipedia.org/wiki/BSA_%28The_Software_Alliance%29

If they're not in the pocket of Microsoft as the BSA is (was?) then 
they're in the pocket of their members. They're not White Knight 
palladins searching for pirated software out of their sense of ethical 
outrage, they're doing it because they're paid to.


> But they have been known to
> litigate and fines of several thousand pounds have been issued to
> infringers(?) But FAST is rarely interested in FOSS software its
> commercial code they worry about. 

Point of order: FOSS *is* commercial code (or at least can be). Just ask 
Red Hat, Apple, Mozilla and even Microsoft.

I think you mean that it is *proprietary* code they care about, and 
even then, I daresay they only litigate on behalf of their clients.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Borrowing restricted code

[Steven D'Aprano]

On Wed, Dec 05, 2018 at 07:49:58PM +, Mark Lawrence wrote:
> On 05/12/2018 16:22, Avi Gross wrote:
> 
> [huge snip]
> 
> Please take yourself to another forum, your ramblings have no place on 
> the *PYTHON TUTOR* mailing list.

Steady on Mark, a lot of what Avi says is misinformed or close to 
stream-of-consciousness irrelevant chatter (no offense Avi, but you do 
go on and on with no apparent point sometimes), but some of it is 
relevant to Python programming.

Every programmer who distributes or copies software needs to have at 
least a basic understanding of copyright and licencing.

We allow discussions about IDEs or the best graphical libraries, and 
they too are only peripherally related to Python. But they are related, 
just as licencing and copyright are.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Borrowing restricted code

[Steven D'Aprano]

On Wed, Dec 05, 2018 at 11:22:35AM -0500, Avi Gross wrote:

> I am NOT advocating copying code. Not even "free" code.
> 
> I am saying there may be times you want to package the code for special
> purposes.

"Packaging" the code IS copying the code.


> Perhaps someone can enlighten me on a subtle aspect here.
> 
> What does it mean to include something?

https://en.wiktionary.org/wiki/include


[... description of how #include works in C ...]
> The above is not meant to be a precise description but indicates that your
> code was rarely all your own. You could literally set it up so if someone
> else left their code unguarded, you could arrange to grab text or library
> code into your finished executable.

The precise details of what an individual programming language means by 
"including" code will, naturally, depend on the language in question.

This is not really the place to go into a detailed description of all 
the possible variations (this is supposed to be a *Python* forum after 
all) but briefly there are at least two kinds of code which we might 
"include": source code and compiled object code.

When "including" source code, the interpreter might read a command like 
"load spam", look up a file "spam", read the contents into memory, parse 
it and run it through the interpreter as if it had been copied and 
pasted into the original file in place of the "load" line.

When "including" object code, the compiler (or often a separate program 
called a linker) can make a copy of the object code and insert that code 
directly into the object code it is generating. This is called "static 
linking".

An alternative is to leave the object code where it is, but instead 
insert instructions for how to access it at runtime. This is called 
"dynamic linking".

https://kb.iu.edu/d/akqn

Python's import is (I think) closer to dynamic linking than the others.


> In python, you can generally have access to the python code of what you can
> import or at least to the byte code. But there may be different rules
> attached to several categories.
> 
> If you use "import" you don't so much copy as USE the code. I mean the
> interpreter pauses evaluating the current file and opens the one you asked
> for and reads it as if it were your code.

That is not entirely wrong, but its not quite right either.

The action of the import command is surprisingly complex, and so I may 
have got some subtle details wrong. But the high-level overview of what 
the interpreter does when you run "import spam" is as follows.

1. Look for an already loaded module "spam"; if the interpreter finds 
one, it creates an new variable called "spam" in the current namespace, 
and assigns that module object to that name. The import is complete.

# Pseudo-code
if "spam" in sys.modules:
spam = sys.modules["spam"]
return


2. If no such already loaded module, then search a set of known 
locations for a library called "spam":

# Pseudo-code
for location in sys.path:
for filename in os.listdir(location):
name, ext = path.splitext(filename)
if name == "spam":
if ext == ".py":
read the source code from spam.py into memory
parse it into bytecode into memory
write out the bytecode to spam.pyc
elif ext == ".pyc":
read the bytecode from spam.pyc into memory
else:
# other cases handled here, e.g. packages, zip files,
# C libraries (.dll or .so), other extensions etc.
# Assuming we get to here...
create a module object in memory
run that bytecode, using that module object as the namespace
cache the module object in sys.modules['spam']
spam = module object
return
# no such "spam" module or package found
raise ImportError


So you can see that modules are only executed the first time the 
interpreter imports them, not on subsequent imports.

There really isn't a distinction to make between code treated "as if it 
were your code" and other code. All code is treated the same.

How could it not be? The interpreter cannot know which modules or 
libraries you wrote, which were collaborative efforts between you and 
other people, and which were written by other people.



> Some actions are transient. A line
> like "5+3" evaluates to 8 and you ignore it. But many and perhaps most lines
> end up creating instantiations of objects (yes, classes are also objects)
> which then sit in memory.

Or get used and then disposed of by the garbage collector.


> Functions are also objects and pretty much contain
> within them everything you need to reconstruct the function if you know
> where to look. Is what is copied really different than the original text
> used? 

Yes.


> If you import the base modules that come with python, can you assume it is
> freely given with few strings other than the kind discussed?

There is no need to assume anything, you 

Re: [Tutor] Any 'graphical' ways of learning Python

[Steven D'Aprano]

On Wed, Dec 05, 2018 at 11:39:49AM +1100, Matthew Polack wrote:
> Hi All,
> 
> We're using Python with our Year 9 and 10 students to teach programming.

Yay! And I see you're a fellow Aussie :-)


> I've started with basic console programming...doing simple games like a
> quiz game etc.
> 
> Students always seem to like 'graphics'..one of the reasons things like
> 'Scratch' are so popular in schools is because of  the ready made GUI.

Indeed.

Alas, graphics is one of Python's weaknesses, especially at the 
beginners' level.

The easiest to use will be the turtle module, although its fairly 
limited in what it does. Its basically a simulation of the turtle 
from the Logo programming language. Logo was cutting edge about half a 
century ago.

You might try PyGame, although I don't know how easy it will be for you 
to pre-install it for the students, or how difficult it is to use (I've 
never used it myself).

https://www.pygame.org/


Another option is PySimpleGUI:

https://pypi.org/project/PySimpleGUI/




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Beginners Book, Python and PyScripter

[Steven D'Aprano]

On Wed, Dec 05, 2018 at 08:44:14AM +, Alan Gauld via Tutor wrote:
> On 04/12/2018 23:52, James Stapleton-Cotton wrote:
> 
> > USERs-MacBook-Pro:~ user$ python hello.py
> > python: can't open file 'hello.py': [Errno 2] No such file or directory
> 
> You need to provide the full path to the Python file.
> 
> > I previously created a python file called 'hello.py' using the PyCharmEdu
> > editor (as instructed). This file is saved in my 'Documents' in a folder
> > called 'Pycharm Projects' 
> 
> So you would have needed to type
> 
> user$ python "~/Documents/Pycharm Projects/hello.py"

To be absolutely clear here, the "user$" part is the prompt, you don't 
type that.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Copyleft

[Steven D'Aprano]

Avi,

Why do you keep changing the subject line? You started a new thread 
about "borrowing" (copying) code, people responded to that thread, and 
you've immediately changed the subject to "Copyleft" even though you 
talk about much more than copyleft licences.

(You barely mention copyleft at all -- and what you say about it is 
very inaccurate.)


On Tue, Dec 04, 2018 at 07:50:21PM -0500, Avi Gross wrote:
> Alan,
> 
> Just a reminder. I am NOT suggesting the first way to solve a problem 
> is what I described. I am saying that sometimes a roadblock (real or 
> imagined) can be gotten around using methods like that.

In my experience, the sorts of organisations which make it difficult to 
install third-party software (whether FOSS or proprietary) are also the 
sorts of organisations which will fire you immediately for copying 
someone else's code.

YMMV.


> Agreed. There are copyrights that have to be considered, albeit often 
> SMALL amounts are considered OK. Attribution is a good idea in 
> general.

There's another factor to consider: software patents. If you're going to 
copy a small amount of code, you may still violate their patent. 
(Disclaimer: I am not a patent lawyer.) In that case, giving credit to 
them will be tantamount to admitting to willful infringement, which in 
most jurisdictions that I know of receives triple damages.

That's how broken the patent system is: the penalties for infringement 
encourage people to avoid doing a patent search. Its better to say "We 
couldn't be bothered doing a patent search" than to say "We tried to do 
due diligence but failed to spot this patent", because when it comes to 
patents, if you make the effort but fail, you are deemed to have 
willfully infringed, whereas if you intentionally make no attempt to 
avoid infringing, your infringement is deemed to have been an innocent 
mistake.

Crazy but true.



> The problem that began this is when someone puts roadblocks in the way of 
> what works.

There may be very good reasons for this policy that you call a 
roadblock.

And we only have the Original Poster's word that he is not allowed 
to install pexpect. That could be his misunderstanding, or perhaps he 
can't be bothered going through the process to get approval.

In fairness, some places do make that process rather gruelling... in a 
previous position, I worked with a client whose change request process 
for installing new software on a production server could easily take 
three or four months for approval to come through, if it was given at 
all. Now, you might be tempted to roll your eyes and call that 
bureaucracy gone mad, but they had very good reasons for that level of 
caution and conservativeness.

A colleague got a figurative spanking after he accidentally brought 
their production server down (fortunately only for ten minutes, although 
that was ten minutes too long) by rolling out a buggy "bug fix" directly 
onto the production server without testing it on a staging server first, 
or getting approval for the change.

https://i.chzbgr.com/full/7756111616/hED5FBCC9/


 
> If someone for example wants to be able to use something as trivial as 
> math.pi but for some reason won't load the math module, you could 
> suggest they look up the value of pi to umpteen decimal places and 
> create an object called math Perhaps a class) containing nothing but a 
> value called pi 

[...]
> Or, I suggest you can copy the real math module into the same place. 
> You may want to edit it with a comment.

So rather than adding a single constant, you're going to risk infringing 
the copyright and moral rights of other programmers by copying a 75 
kilobyte C library that you don't need, just to get the value of pi 
which you could legally and easily copy from your calculator or Google 
in about five seconds.

Are you being paid by the hour? *wink*


> But, yes, bad example as pretty much any implementation of python will 
> include the math module as it is standard.

Indeed. Let's take a better example: the pexpect library.

The OP is not able (not permitted? not physically able? can't be 
bothered?) to install the pexpect library. So your advice for him is 
to... install the pexpect library, potentially ignoring its licence 
terms, making a copy of the source (hoping that its all Python and not C 
that needs compiling) and edit the source to "add a comment" (saying 
what?).

I think you haven't really considered that if the OP is not able to 
install the pexpect library, he probably won't be able to copy it into 
his application either. To say nothing of the ethical issues of copying 
software without obeying the licence terms.

Fortunately pexpect itself is licenced under a very permissive licence 
that allows copying with (almost) no restriction:

https://github.com/pexpect/pexpect/blob/master/LICENSE

but as a general matter of principle, suggesting people just copy the 
source code they want and stick it in their application is rather risky 
and 

Re: [Tutor] Borrowing free code

[Steven D'Aprano]

On Tue, Dec 04, 2018 at 11:43:05PM +, Alan Gauld via Tutor wrote:
> On 04/12/2018 19:31, Avi Gross wrote:
> 
> > But some packages are simply python code that you can 
> > simply insert into your own python files. 
> 
> If they are fully public domain that's probably true.

Almost nothing younger than 70 years is in the public domain. Nearly all 
countries share certain copyright laws, including automatic granting of 
copyright for the life of the author + many years (exactly how many 
depends on the country) whether you want it or not.

According the lawyers behind the Creative Commons, many jurisdictions 
make it difficult if not impossible to voluntarily give up copyright and 
put a work in the public domain:

https://creativecommons.org/share-your-work/public-domain/cc0/

One of the few exceptions is that many (but not all) works by the US 
government are required to be placed in the public domain by law.

Having said that, all is not *quite* lost (yet). There are still ways to 
get equivalent freedom as if the work was in the public domain, and 
another Mickey Mouse Copyright Extension Act is unlikely to occur any 
time soon.


> If they are copyright (even if open/free) you would be
> potentially liable for prosecution since you are copying
> someone else's work.

That's a rather negative way of looking at it.

Copyright infringement is typically handled through the civil courts: 
the legal authorities don't press charges for copying (except, perhaps, 
in the case of large-scale piracy of manufactured goods like fake 
designer handbags, movie DVDs, etc). Rather, the copyright owner has to 
sue you. If the copyright owner is explicitly giving you the right to 
copy and modify the software, which is the case for FOSS (Free Open 
Source Software), then you have nothing to fear so long as you have a 
valid licence.

If you have a valid licence to use and copy the software, and you obey 
the licence terms, then you are in no danger of being prosecuted for 
copyright infringement because you are licenced to do so.


> Even if it is open source then at the very least
> you should include a comment to the effect that
> the code is based on, say, M Palin's file parrot.py
> or whatever.

Giving credit might be good from an ethical point of view, but it may 
not be either necessary or sufficient (depending on the licence).

Some examples: 

The MIT and BSD licences do not explicitly require you to credit the 
author, but they do require you to include a copy of the original 
author's copyright and licence in your work:

https://en.wikipedia.org/wiki/BSD_licenses
https://en.wikipedia.org/wiki/MIT_License

If you choose to give credit as a courtesy, that's fine, but you must be 
careful as to do so in such a way that you avoid giving the impression 
that the author endorses or is responsible for your work.

Good: "MyApp, by me, with a big thank you to J. Cleese for 
his spam library."

Bad: "MyApp with spam by J. Cleese!"


On the other hand, the GPL requires more than just credit, it requires 
that (with some exceptions, including fair use) if you duplicate their 
code, that your code in turn must be released under the GPL as well. 
Merely giving credit is in no way sufficient.

And on the gripping hand, you have ultra-permissive public-domain like 
licences such as the creative commons CC0 or the Toybox licence:

https://en.wikipedia.org/wiki/Public-domain-equivalent_license

neither of which require credit be given.

Regardless of the licence, you must obey the conditions (whether they 
are minimal, as in the MIT licence, or onerous and difficult as in most 
proprietary licences) in order to be legally permitted to copy the work. 
But if you have a licence to copy, then of course you may copy according 
to the terms of the licence.

In a practical sense, copying trivial amounts of code from software 
licenced under the MIT licence, or similar, would be highly unlikely to 
get you sued. For starters, the author would have to know you copied it, 
and care, and track you down in real life, and sue you. And if the code 
snippet was small enough, you could defend on the basis of fair use.

(Although more and more jurisdictions are taking a hard-line, minimalist 
approach to fair use, allowing it effectively only for the purposes of 
satire. Or even not acknowledging a fair use right at all.)

Another factor may be that, regardless of the *actual* legal risk, some 
employers may not allow copying or use of (some or all) FOSS software 
even if licenced, because they may fear the unknown, or they believe 
anti-GPL propoganda ("its a virus that means you will have to give up 
your intellectual property"), ethical reasons (copying without 
attribution may be plagiarism), or possibly even for legitimate legal 
reasons ("the GPL is not compatible with our licence").

Bottom line: yes, you can legally copy FOSS software, under certain 
conditions, and need not fear being prosecuted.

(Actually, you could