Re: unintuitive for-loop behavior
Earlier, I wrote:
> On Sat, 1 Oct 2016 10:46 am, Gregory Ewing wrote:
[...]
>> Whenever there's binding going on, it's necessary to decide
>> whether it should be creating a new binding or updating an
>> existing one.
>
> Right.
I changed my mind -- I don't think that's correct.
I think Greg's suggestion only makes sense for languages where variables are
boxes with fixed locations, like C or Pascal. In that case, the difference
between creating a new binding and updating a new one is *possibly*
meaningful:
# create a new binding
x: address 1234 > [ box contains 999 ]
x: address 5678 > [ a different box, containing 888 ]
What happens to the old x? I have no idea, but the new x is a different box.
Maybe the old box remains there, for any existing code that refers to the
address of (old) x. Maybe the compiler is smart enough to add address 1234
to the free list of addresses ready to be used for the next variable. Maybe
its just lost and unavailable until this function exists.
# update an existing binding
x: address 1234 > [ box contains 999 ]
x: address 1234 > [ same box, now contains 888 ]
That's the normal behaviour of languages like C and Pascal. But its distinct
from the previous, hypothetical behaviour.
But Python doesn't work that way! Variables aren't modelled by boxes in
fixed locations, and there is no difference between "create a new binding"
and "update an existing one". They are indistinguishable. In both
cases, 'x' is a key in a namespace dict, which is associated with a value.
There's no difference between:
x = 999
del x
x = 888
and
x = 999
x = 888
If you consider the namespace dict as a hash table, e.g. something like this
(actual implementations may differ):
[ UNUSED, UNUSED, (key='y', value=23), (key='a', value=True),
UNUSED, (key='x', value=999), UNUSED ]
then binding 888 to 'x' must put the key in the same place in the dict,
since that's where hash('x') will point.
Subject to linear addressing, chaining, re-sizes, or other implementation
details of hash tables of course. But all else being equal, you cannot
distinguish between the "new binding" and "update existing binding"
cases -- in both cases, the same cell in the hash table gets affected,
because that's where hash('x') points. If you put it somewhere else, it
cannot be found.
So I think I was wrong to agree with Greg's statement. I think that for
languages like Python where variables are semantically name bindings in a
namespace rather than fixed addresses, there is no difference between
updating an existing binding and creating a new one.
In a language like Python, the only distinction we can make between name
bindings is, which namespace is the binding in? In other words, what is the
current block of code's scope?
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sat, Oct 1, 2016 at 5:06 PM, Steve D'Aprano
wrote:
> Earlier, I wrote:
>
>> On Sat, 1 Oct 2016 10:46 am, Gregory Ewing wrote:
> [...]
>>> Whenever there's binding going on, it's necessary to decide
>>> whether it should be creating a new binding or updating an
>>> existing one.
>>
>> Right.
>
> I changed my mind -- I don't think that's correct.
>
> I think Greg's suggestion only makes sense for languages where variables are
> boxes with fixed locations, like C or Pascal. In that case, the difference
> between creating a new binding and updating a new one is *possibly*
> meaningful:
>
> # create a new binding
> x: address 1234 > [ box contains 999 ]
> x: address 5678 > [ a different box, containing 888 ]
>
> What happens to the old x? I have no idea, but the new x is a different box.
> Maybe the old box remains there, for any existing code that refers to the
> address of (old) x. Maybe the compiler is smart enough to add address 1234
> to the free list of addresses ready to be used for the next variable. Maybe
> its just lost and unavailable until this function exists.
>
> # update an existing binding
> x: address 1234 > [ box contains 999 ]
> x: address 1234 > [ same box, now contains 888 ]
>
>
> That's the normal behaviour of languages like C and Pascal. But its distinct
> from the previous, hypothetical behaviour.
>
> But Python doesn't work that way! Variables aren't modelled by boxes in
> fixed locations, and there is no difference between "create a new binding"
> and "update an existing one". They are indistinguishable. In both
> cases, 'x' is a key in a namespace dict, which is associated with a value.
> ...
> In a language like Python, the only distinction we can make between name
> bindings is, which namespace is the binding in? In other words, what is the
> current block of code's scope?
Yes, this is true; however, it's not difficult to create subscopes. I
put together a POC patch a while ago to make 'with' blocks able to put
their capture names into a separate subscope, transparently. I don't
think it's what Python wants, but it would be reasonably coherent
semantically ("with expr as name:" puts name, and name alone, into a
subscope - any other assignments are not), and wasn't difficult to
implement in CPython.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sat, 1 Oct 2016 02:39 am, Chris Angelico wrote:
> On Sat, Oct 1, 2016 at 12:36 AM, Grant Edwards
> wrote:
>> In C99 a for loop has its own namespac:
[...]
> I believe that's the same semantics as C++ uses, and I agree, it's
> very convenient. Among other things, it means that nested loops behave
> more like they do in Python, with independent iterators:
*scratches head*
So let me see if I understand this...
You're suggesting that C99, where for-loops have their own namespaces, is
MORE like Python (where for-loops DON'T have their own namespace), than
C89, which, like Python, DOESN'T give for-loops their own namespace?
That's ... curious.
I'm not saying you're wrong: after spending three quarters of an hour trying
to fix a six line (including one blank line) C program because I
accidentally closed a comment with /* instead of */, I will believe
anything about C[1]. If you tell me that void causes birth defects and
printf is responsible for the police shootings of unarmed black men, I'll
believe every word of it.
> int main(void)
> {
> for (int i=0; i<5; ++i)
> {
> printf("%d:", i);
> for (int i=0; i<3; ++i)
> printf(" %d", i);
> printf("\n");
> }
> }
>
> Now, granted, this is not something I would ever actually recommend
> doing, and code review is absolutely justified in rejecting this...
So let me see if I understand your argument... for-loop namespaces are good,
because they let you write code that you personally wouldn't write and
would, in fact, reject in a code review.
O-kay.
> but it's a lot better than pure function-scope variables, where you'd
> get stuck in an infinite loop.
That's the risk that you take when you have a C-style for loop and you
modify the loop variable. There's nothing special about the inner loop in
that regard:
#include /* for printf */
int main(void)
{
for (int i=0; i<5; ++i)
{
printf("%d:", i);
i = 0;
}
}
Solution: don't do that.
[1] Apparently useful error messages are one of those things C programmers
eschew, like type safety, memory safety, and correctness.
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Steve D'Aprano writes: > In a language like Python, the only distinction we can make between > name bindings is, which namespace is the binding in? In other words, > what is the current block of code's scope? Python already has nested scopes. A local scope for just those variables introduced in for-loops could be carved out as in the following example. The following example function has three parameters and two local variables, one of which is only assigned inside the loop (and it also references one global variable), in addition to the variable introduced in the loop. def eg(wev, a, b): y = 0 for x in wev: # to be given a new semantics f(a, x) f(x, b) y += 1 z = (y > 12) return y, z Replace the loop with a call to a thunk that refers to the variables in the outer scopes, except for the one variable that the loop introduces - that variable is to be local to the thunk. So the example function should be equivalent to something like the following translation. def eg(wev, a, b): y = 0 def g31(): nonlocal y, z # the assigned variables g32 = iter(wev) while 1: # need to catch StopIteration x = next(g32) # now local to g31 f(a, x) f(x, b) y += 1 z = (y > 12) g31() return y, z A different translation would introduce each x in a different function for each step in the iteration. That's just a variation on the overall theme. -- https://mail.python.org/mailman/listinfo/python-list
generator no iter - how do I call it from another function
Evening
My file list handler I have created a generator.
Before I created it as a generator I was able to use iter on my lxml root
objects, now I cannot iter.
± |master U:2 ?:1 ✗| → python3 race.py data/ -e *.xml
Traceback (most recent call last):
File "race.py", line 83, in
dataAttr(rootObs)
File "race.py", line 61, in dataAttr
for meet in roots.iter("meeting"):
AttributeError: 'generator' object has no attribute 'iter'
How do I now pull the next iterations through from the other function?
from lxml import etree
import csv
import re
import argparse
import os
parser = argparse.ArgumentParser()
parser.add_argument("path", type=str, nargs="+")
parser.add_argument(
'-e', '--extension', default='', help='File extension to filter by.')
# >python race.py XML_examples/ -e .xml
args = parser.parse_args()
name_pattern = "*" + args.extension
my_dir = args.path[0]
for dir_path, subdir_list, file_list in os.walk(my_dir):
for name_pattern in file_list:
full_path = os.path.join(dir_path, name_pattern)
def return_files(file_list):
"""
Take a list of files and return file when called.
Calling function to supply attributes
"""
for filename in sorted(file_list):
with open(dir_path + filename) as fd:
tree = etree.parse(fd)
root = tree.getroot()
yield root
def clean(attr):
"""
Split list into lists of 5 elements.
if list is less than 5 in length then a 0
is appended and the list returned
"""
p = re.compile('\d+')
myList = p.findall(attr)
if len(myList) < 5:
myList.append('0')
return myList[0], myList[1], myList[2], myList[3], myList[4]
def dataAttr(roots):
"""Get the root object and iter items."""
with open("output/first2.csv", 'w', newline='') as csvf:
race_writer = csv.writer(csvf, delimiter=',')
for meet in roots.iter("meeting"):
print(meet)
for race in roots.iter("race"):
for nom in roots.iter("nomination"):
meetattr = meet.attrib
raceattr = race.attrib
nomattr = nom.attrib
if nomattr['number'] != '0':
firsts = clean(nomattr['firstup'])
race_writer.writerow(
[meetattr['id'], meetattr['date'],
meetattr['venue'], raceattr['id'],
raceattr['number'], raceattr['distance'],
nomattr['id'], nomattr['barrier'],
nomattr['weight'], nomattr['rating'],
nomattr['description'], nomattr['dob'],
nomattr['age'], nomattr['decimalmargin'],
nomattr['saddlecloth'], nomattr['sex'],
firsts[4]])
rootObs = return_files(file_list)
dataAttr(rootObs)
--
https://mail.python.org/mailman/listinfo/python-list
Re: rocket simulation game with just using tkinter
On Saturday, 1 October 2016 08:59:28 UTC+10, Irmen de Jong wrote: > Hi, > > I've made a very simple rocket simulation game, inspired by the recent > success of SpaceX > where they managed to land the Falcon-9 rocket back on a platform. > > I was curious if you can make a simple graphics animation game with just > using Tkinter, > instead of using other game libraries such as PyGame. > As it turns out, that works pretty well and it was quite easy to write. > Granted, there's > not much going on on the screen, but still the game runs very smoothly and I > think it is > fun for a little while where you try to learn to control the rocket and > attempt to > successfully land it on the other launchpad! > > The physics simulation is tied to the game's frame rate boohoo, but the > upside is that > you can change the framerate to control the game's difficulty. It's easy at > <=20, fun at > 30 and impossible at 60 :) It's running on 30 by default. > > > You can get the code here if you want to give it a try: > https://github.com/irmen/rocketsimulator > > So you just need python 2/3 with tkinter to play this! > > > Have fun > Irmen Well done. An interesting listen that might be up your alley, how-i-built-an-entire-game-and-toolchain-100-in-python on talkpython https://talkpython.fm/episodes/show/78/how-i-built-an-entire-game-and-toolchain-100-in-python Sayth -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sat, Oct 1, 2016 at 6:35 PM, Steve D'Aprano
wrote:
> On Sat, 1 Oct 2016 02:39 am, Chris Angelico wrote:
>
>> On Sat, Oct 1, 2016 at 12:36 AM, Grant Edwards
>> wrote:
>
>>> In C99 a for loop has its own namespac:
> [...]
>
>> I believe that's the same semantics as C++ uses, and I agree, it's
>> very convenient. Among other things, it means that nested loops behave
>> more like they do in Python, with independent iterators:
>
> *scratches head*
>
> So let me see if I understand this...
>
> You're suggesting that C99, where for-loops have their own namespaces, is
> MORE like Python (where for-loops DON'T have their own namespace), than
> C89, which, like Python, DOESN'T give for-loops their own namespace?
>
> That's ... curious.
Well, when you put it like that, it does sound very curious!!
Here's how I read it:
# Python
def main():
for i in range(5):
print(i, end=": ")
for i in range(3):
print(i, end=" ")
print("")
/* Classic C */
int main() {
int i;
for (i=0; i<5; ++i) {
printf("%d:", i);
for (i=0; i<3; ++i)
printf(" %d", i);
printf("\n");
}
//C++ or C99
int main() {
for (int i=0; i<5; ++i) {
printf("%d:", i);
for (int i=0; i<3; ++i)
printf(" %d", i);
printf("\n");
}
The first and last have *independent iterators*. When you complete the
inner loop, the outer loop picks up where *it* left off, not where you
happen to have reassigned the iteration variable. The middle one is an
infinite loop.
> I'm not saying you're wrong: after spending three quarters of an hour trying
> to fix a six line (including one blank line) C program because I
> accidentally closed a comment with /* instead of */, I will believe
> anything about C[1]. If you tell me that void causes birth defects and
> printf is responsible for the police shootings of unarmed black men, I'll
> believe every word of it.
Did you turn all warnings on? Look for a "pedantic" mode. You'll get a
lot more information, assuming you're using a modern decent C
compiler. (GCC, for instance, will even warn about situations where it
looks like the braces and indentation don't match, thus bringing a
Python feature to C, more-or-less.)
>> int main(void)
>> {
>> for (int i=0; i<5; ++i)
>> {
>> printf("%d:", i);
>> for (int i=0; i<3; ++i)
>> printf(" %d", i);
>> printf("\n");
>> }
>> }
>>
>> Now, granted, this is not something I would ever actually recommend
>> doing, and code review is absolutely justified in rejecting this...
>
> So let me see if I understand your argument... for-loop namespaces are good,
> because they let you write code that you personally wouldn't write and
> would, in fact, reject in a code review.
>
> O-kay.
Yes. In the first place, I wouldn't necessarily reject this code; it's
a minor issue only, thanks to the namespacing and nested scoping. It
does cost a little in clarity, and I would point it out to a student,
but in less trivial examples than this, it's really not that big a
deal.
*Without* namespacing to keep them apart, this is a serious problem.
Thanks to namespacing, it's only a minor problem (you've shadowed one
variable with another). It's comparable to this Python code:
def send_message(from, to, subj, body):
str = "Mail: From " + from
str += "\nRecipient: To " + to
str += "\nSubject: " + subj
str += "\n\n" + body
send_raw_data(str)
Would you reject this in a code review? Maybe. I wouldn't yell at you
saying "shadowing is fine, you have no right to reject that". But it's
also not inherently a problem. The rules of name shadowing are
well-defined (in both languages) and designed to *permit*, not reject,
constructs like this.
>> but it's a lot better than pure function-scope variables, where you'd
>> get stuck in an infinite loop.
>
> That's the risk that you take when you have a C-style for loop and you
> modify the loop variable. There's nothing special about the inner loop in
> that regard:
>
>
> #include /* for printf */
> int main(void)
> {
> for (int i=0; i<5; ++i)
> {
> printf("%d:", i);
> i = 0;
> }
> }
>
>
> Solution: don't do that.
Actually, I would call this version a feature. If you deliberately and
consciously reassign the iteration variable, it's usually because you
wanted to adjust the loop. Classic example: iterating through a series
of tokens, and sometimes needing to grab a second token. Python lets
you do this by explicitly calling iter() and next(); C lets you do
this by incrementing the loop counter (which works only with the
less-flexible looping style it has).
But when you didn't intend to reassign it, it's usually because you
had a local name that happened to collide. Sure, that can happen
anywhere, especially with abbreviated variable names (I once had the
amusing situation of trying to use "cursor" and "current" in the same
namespace, both abbreviated "cur"), b
Re: How to make a foreign function run as fast as possible in Windows?
Chris Angelico at 2016/10/1 11:25:03AM wrote: > What's it doing? Not every task can saturate the CPU - sometimes they > need the disk or network more. > This function has no I/O or similar activity, just pure data processing, and it takes less than 200 bytes of data area to work with. My CPU is an i3 (4 threads/2 cores). I suppose after this job was assigned to run on a particular core, the OS shouldn't bother it by other system related tasks anymore. If it does, won't be this OS designed stupidly? I was puzzled why the core has no 100% usage? Why always has some percentage of idle? --Jach -- https://mail.python.org/mailman/listinfo/python-list
Re: generator no iter - how do I call it from another function
My main issue is that usually its just x in ,,, for a generator.
But if I change the code
for meet in roots.iter("meeting"):
to
for meet in roots("meeting"):
Well its invalid but I need to be able to reference the node, how do I achieve
this?
Sayth
--
https://mail.python.org/mailman/listinfo/python-list
Re: generator no iter - how do I call it from another function
2016-10-01 12:09 GMT+02:00 Sayth Renshaw :
> My main issue is that usually its just x in ,,, for a generator.
>
> But if I change the code
> for meet in roots.iter("meeting"):
>
> to
> for meet in roots("meeting"):
>
> Well its invalid but I need to be able to reference the node, how do I
> achieve this?
>
> Sayth
> --
Hi,
i think, you identified the problem correctly in the previous mail:
>> Before I created it as a generator I was able to use iter on my lxml root
>> objects, now I cannot iter.
It is not clear, what the previous - working - version was, but there
is a difference between using built-in iteration handling (with
built-in iter(...) or implicitly via "for elem in
some_iterable_object: ...") and with the specialised "iter" method of
root in the lxml library (which has specific functionalities).
I don't have much experience with that library, but if your change was
basically adding a "generator" layer around "roots",
you may try to iterate over it to get its elements, which might
supposedly support the method, you were using previously.
e.g. (completely untested, as there is no xml underlying given in your sample)
for root in roots:
for meet in root.iter("meeting"):
...
hth,
vbr
--
https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Random832 wrote: > for a in collection: > b = some_calculation_of(a) > final b: do_something_with(lambda: ... b ...) I would prefer something like for a in collection: let b = some_calculation_of(a): do_something_with(lambda: ... b ...) -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: generator no iter - how do I call it from another function
On Sat, 1 Oct 2016 06:52 pm, Sayth Renshaw wrote: > Evening > > My file list handler I have created a generator. > > Before I created it as a generator I was able to use iter on my lxml root > objects, now I cannot iter. lxml root objects have an iter method. Other objects do not. Thank you for showing your code, but it is MUCH too complicated. Most of the code has nothing to do with your question. You are asking a question about generators, but your code shows argument processing, file processing, regexes, all sorts of things that we have to read and understand before we can answer your question, and none of it is relevant. That's too much work. We are volunteers. If you want to pay us, we'll be happy to spend hours on your code. But if you want free consulting, then you have to do some of the work to make it easy for us. Please cut your code down to the smallest possible amount that shows the problem: http://sscce.org/ If you have a generator, you can iterate over it like this: def generator(): yield 1 yield 2 yield 4 for value in generator(): # NOT generator.iter() print(value) Does that answer your question? If not, then cut your code down to the smallest possible amount that shows the problem: http://sscce.org/ and I'll read it again. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Copying a compiled Python from one system to another
Long story short: I have no working systems capable of compiling the latest Python 3.6, and no time to upgrade my usual machines to something which will work. However I do have access to another machine (actually a VM) which can compile Python 3.6. It's not practical for me to use it as a my main development machine, but as a temporary measure, I thought I could compile 3.6 on this VM, then copy the python binary to my usual desktop machine. What sort of challenges am I likely to find? Both machines are Linux, but different distros. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sat, 1 Oct 2016 01:44 pm, Rustom Mody wrote: > Yes one basic problem with comprehensions in python is that they are > defined by assignment not binding to the comprehension variable ¿Que Mr Fawlty? I'm sorry, I don't understand you. In Python, all assignments are name bindings. So you seem to be saying: one basic problem with comprehensions in python is that they are defined by binding (to the comprehension variable) not binding to the comprehension variable which confuses me. How do you think that comprehension variables are different from all other variables? And while we're at it, what do you mean by "comprehension variable"? Are you talking about the variable that the comprehension is bound to: comprehension_variable = [x+1 for x in sequence] or are you talking about the loop variable inside the comprehension? foo = [comprehension_variable+1 for comprehension_variable in sequence] It is not clear what you mean. Although I suspect it makes no difference -- either way, the nature of the assignment is identical: its a name binding. > Python copied comprehensions from haskell and copied them wrong > Here are all the things (that I can think of) that are wrong: > 1. Scope leakage from inside to outside the comprehension The core devs agree that this was a mistake. It was rectified in generator expressions, and for comprehensions in Python 3. This is no longer an issue. > 2. Scope leakage from one value to the next I don't know what that means. > 3. The name 'for' misleadingly associates for-loops and comprehensions What is misleading about it? > 4. The for-loop based implementation strategy made into definitional > semantics > 5. The absence of simple binding inside comprehensions: >[f(newvar) for v in l newvar = rhs] I don't know what this means. > 1 was considered sufficiently important to make a breaking change from > python2 to 3 Correct. > 2 is what causes the lambda gotcha I don't think so. > 3 is what makes noobs to take longer than necessary to grok them You haven't shown any evidence that beginners take longer than necessary to grok list comprehensions. Perhaps they take exactly as long as necessary. And you CERTAINLY haven't demonstrated that the average beginner would understand Haskell's list comprehensions more easily than Python's. https://wiki.haskell.org/List_comprehension https://www.haskell.org/onlinereport/exps.html#sect3.11 > 4 is what causes a useless distinction between 1 and 2 — scope leakage is > scope leakage. What is scope leakage? > The explanatory mechanisms of why/whither/what etc should > at best be secondary 5. is workaroundable with a [... for newvar in [rhs]] >Possible and clunky I don't know what this means. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Steve D'Aprano wrote: # create a new binding x: address 1234 > [ box contains 999 ] x: address 5678 > [ a different box, containing 888 ] In the context of CPython and nested functions, replace "box" with "cell". When I said "creating a new binding" I meant that the name x refers to different cells at different times. When I said "updating an existing binding" I meant that the name x still refers to the same cell, but that cell refers to a different object. In a wider context, replace "box" with "slot in a stack frame" or "slot in a namespace dictionary". But Python doesn't work that way! Variables aren't modelled by boxes in fixed locations, and there is no difference between "create a new binding" and "update an existing one". There is very much a distintion. Each time you invoke a function, a new set of bindings is created for all of its parameters and local names. Assigning to those names within the function, on the other hand, updates existing bindings. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sat, Oct 1, 2016 at 9:21 PM, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the latest > Python 3.6, and no time to upgrade my usual machines to something which > will work. > > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could compile > 3.6 on this VM, then copy the python binary to my usual desktop machine. > > What sort of challenges am I likely to find? Both machines are Linux, but > different distros. First off, are they the same CPU architecture? If not, you're going to have a lot of hassles. (This includes x86_64 vs i386. You *can* run a 32-bit Python on a 64-bit Linux, but it'll be a pain.) I'm going to assume that they are. Are both Linuxes of broadly similar vintage? The most likely problems you'll face will be library versions - CPython will happily link against any of several compatible versions of a library, but once it's linked against one, it'll want to find the same version thereafter. Worst case, you could symlink, but that could cause trouble when you upgrade, so I'd advise against it. In fact, your best bet might actually be to *minimize* the number of dev libraries on the build machine. You'll have build failures in optional modules (and thus less modules), but less chances of running into difficulties. Other than that, it should be possible. You'll probably want to export the built Python as a package manager file, eg using checkinstall [1] on Debian-based systems, as it'll be a lot easier to pick up and carry around. But it shouldn't be too hard even working manually. ChrisA [1] https://wiki.debian.org/CheckInstall -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Saturday, October 1, 2016 at 5:02:30 PM UTC+5:30, Steve D'Aprano wrote: > On Sat, 1 Oct 2016 01:44 pm, Rustom Mody wrote: > > > Yes one basic problem with comprehensions in python is that they are > > defined by assignment not binding to the comprehension variable > > ¿Que Mr Fawlty? > > I'm sorry, I don't understand you. > > In Python, all assignments are name bindings. So you seem to be saying: > > one basic problem with comprehensions in python is that they > are defined by binding (to the comprehension variable) not > binding to the comprehension variable > > which confuses me. How do you think that comprehension variables are > different from all other variables? Sorry... I guess you are right [for once ;-) ] I was making a *generic* distinction between binding-constructs that *create* variables, eg function-defs, lambdas, excepts, withs etc and assignment that *changes* a binding. This distinction is standard in compiled languages like C where a int x; creates the x and x=rhs; modifies it C++ preserves this distinction in user defined types making sure that the constructor called for an assignment and for an initialization are distinct even though they look very similar: x = rhs; vs T x = rhs; In functional languages, both the dynamic ones like lisp as well as the static ones like Haskell are very strict and sacrosanct about this distinction. However in python this does not work because assignment both assigns and creates the variable. Interestingly there is this thread running right now on haskell-cafe: https://groups.google.com/forum/#!topic/haskell-cafe/6tqMdy9nGdc which is inspired by python's “batteries included” And there there is this comment: «Python is even more imperative than C++ or Java, it's dynamically typed…» which (I guess) is because of exactly such features Long story short: My saying “binding” is meaningless in python. I should have said function-binding; ie the mechanism by which functions bind their arguments *anew* And then define comprehensions not as now done in terms of for loops that mutatingly extend the list being built up but as recursive functions that get (re)called for every new value of the comprehension variable passed and therefore fresh-bound as parameter > > And while we're at it, what do you mean by "comprehension variable"? Are you > talking about the variable that the comprehension is bound to: > > comprehension_variable = [x+1 for x in sequence] > > or are you talking about the loop variable inside the comprehension? > > foo = [comprehension_variable+1 for comprehension_variable in sequence] > Yeah by comprehension-variable I mean the one that sits left of the ‘in’ inside The other obviously needn't exist -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sat, 1 Oct 2016 10:01 pm, Chris Angelico wrote: >> [...] I thought I could >> compile 3.6 on this VM, then copy the python binary to my usual desktop >> machine. >> >> What sort of challenges am I likely to find? Both machines are Linux, but >> different distros. > > First off, are they the same CPU architecture? I'll say yes. > Are both Linuxes of broadly similar vintage? That depends on what you mean by "broadly similar". As far as I am concerned, a five year difference is not very much, and is broadly similar -- it's not like I'm using Linux from 1991. But the whole point is that I need something with gcc 4.8 (maybe 4.7 will do, not sure) but certainly not 4.4 which is the most recent I can get on my current systems. > The most likely problems > you'll face will be library versions - CPython will happily link > against any of several compatible versions of a library, but once it's > linked against one, it'll want to find the same version thereafter. > Worst case, you could symlink, but that could cause trouble when you > upgrade, so I'd advise against it. Does gcc support static linking? Even if I end up with a much bigger binary, at least I know it will have everything it needs to run and I won't have to deal with DLL hell. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sat, 1 Oct 2016 09:33 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: > >> # create a new binding >> x: address 1234 > [ box contains 999 ] >> x: address 5678 > [ a different box, containing 888 ] > > In the context of CPython and nested functions, replace > "box" with "cell". Cells in Python are an implementation detail. It only applies to CPython (as far as I know), certainly not to IronPython, and the interpreter takes care to ensure that the user-visible behaviour of local variables in cells is identical to the behaviour of name bindings in dicts. The only user-visible semantic differences between variables in a dict and variables in cells are: - writing to locals() won't necessarily affect the actual locals; - in Python 3 only, import * and exec in the local names space may be prohibited. Everything else (that I know of) is indistinguishable. Python takes care to hide the differences, for example even though local variables have a cell pre-allocated when the function is called, trying to access that cell before a value is bound to the local gives a NameError (UnboundLocalError) rather than accessing uninitialised memory, as a naive implementation might have done. In any case, I think this is going off on a rather wide tangent -- how is this specifically relevant to the "unintuitive for-loop behavior" the OP was surprised by? > When I said "creating a new binding" I meant that the > name x refers to different cells at different times. > When I said "updating an existing binding" I meant that > the name x still refers to the same cell, but that cell > refers to a different object. I don't believe that there is any test you can write in Python that will distinguish those two cases. (Excluding introspection of implementation details, such as byte-code, undocumented C-level structures, etc.) As far as ordinary Python operations go, I don't see that there's any difference between the two. When you say: x = 0 x = 1 inside a function, and the interpreter does the name binding twice, there's no way of telling whether it writes to the same cell each time or not. Apart from possible performance and memory use, what difference would it make? It is still the same 'x' name binding, regardless of the implementation details. > In a wider context, replace "box" with "slot in a > stack frame" or "slot in a namespace dictionary". > >> But Python doesn't work that way! Variables aren't modelled by boxes in >> fixed locations, and there is no difference between "create a new >> binding" and "update an existing one". > > There is very much a distintion. Each time you invoke > a function, a new set of bindings is created for all of > its parameters and local names. Assigning to those names > within the function, on the other hand, updates existing > bindings. Certainly when you call a function, the local bindings need to be created. Obviously they didn't exist prior to calling the function! I didn't think that was the difference you were referring to, and I fail to see how it could be relevant to the question of for-loop behaviour. As I understood you, you were referring to assignments to *existing* names. If a binding for x already exists, then and only then do you have a choice between: - update the existing binding for x; - or create a new binding for x. If there is no binding for x yet (such as before the function is called), then you have no choice in the matter: you cannot possibly update what doesn't exist. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Rustom Mody writes:
> And then define comprehensions not as now done in terms of for loops
> that mutatingly extend the list being built up but as recursive
> functions that get (re)called for every new value of the comprehension
> variable passed and therefore fresh-bound as parameter
You'd get the magical semantics for comprehensions from the current
definition of comprehension semantics in terms of the loop, if the loop
semantics was magical. Let me demonstrate. (The b-word is unfortunately
not available, so I substitute "magical" instead.)
# delayd = [ lambda : c for c in "abracadabra" ]
# is roughly/almost equal to the following.
delayd = []
for c in "abracadabra":
delayd.append(lambda : c)
print('Python:', ''.join(f() for f in delayd))
# But that for-loop could have been roughly equal to the following,
# giving both the comprehension and the underlying for-loop a
# semantics that some people say they would prefer.
delayd = [] # reset the list, not part of the loop
g1 = iter("abracadabra")
try:
while True:
def g2(c):
delayd.append(lambda : c)
g2(next(g1))
except StopIteration:
pass
print('Magick:', ''.join(f() for f in delayd))
# Output from the above:
# Python: aaa
# Magick: abracadabra
--
https://mail.python.org/mailman/listinfo/python-list
announcing fython
Hi All, I would like to announce a new project call Fython. In short, Fython is Fortran with a Python syntax. Fython permits to write numerical code with the same syntax then Python. Under the hood, the code is transpiled to Fortran and run at top speed. Fython primary goal is to facilitate numerical coding: - Fython code can be easily shared, because a Fython package behaves as a regular Python package. - Fython offers many syntaxtic sugars, such as template and automatic variable completion, increasing programmer productivity. - Fython assists for debugging, and can report the precise line where a segfault happen. Compared to Cython, Fython is dedicated to numerical computing. This dedication produces often clearer and faster code. Compared to Julia, Fython compiles directly to machine instructions. The programmer has total control over the compiled instructions, and this is desirable in some situations. Fython is alpha code, but the code is sufficently stable to be used. I think Fython has the potential to integrate the Python scientific stack. I hope some people will try it and provide feedback on how best to reach this goal. Fython can be find here: fython.readthedocs.io Thanks, Nicolas -- https://mail.python.org/mailman/listinfo/python-list
Re: rocket simulation game with just using tkinter
Am 01.10.16 um 00:59 schrieb Irmen de Jong: Hi, I've made a very simple rocket simulation game, inspired by the recent success of SpaceX You can get the code here if you want to give it a try: https://github.com/irmen/rocketsimulator Nice! I'll have to rebind the keys before I can successfully play this, though. My [] "keys" are in fact combinations of Alt+5 and Alt+6 on a German Macbook keyboard (on a German PC, it is AltGr+8, AltGr+9). Why not using the arrow keys? These should be pretty universal Christian So you just need python 2/3 with tkinter to play this! Have fun Irmen -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On 2016-10-01, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the > latest Python 3.6, and no time to upgrade my usual machines to > something which will work. > > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could compile > 3.6 on this VM, then copy the python binary to my usual desktop machine. You'll probably need to copy more than just the binary. > What sort of challenges am I likely to find? Missing or incompatible libraries. Wrong CPU type. > Both machines are Linux, but different distros. If they were the same distros, you'd have a much better chance. Then, the right way to do it would be to build a binary package (.rpm, .deb, .whatever) on one machine for installation on the other machine using the normal package manager. That way the library situation would get verified (if perhaps not remedied). -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On 2016-10-01, Steve D'Aprano wrote: > Does gcc support static linking? Yes, but the real question is the CPython makefile includes recipes for a statically-linked target. > Even if I end up with a much bigger binary, at least I know it will > have everything it needs to run and I won't have to deal with DLL > hell. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: generator no iter - how do I call it from another function
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
Loving life.
I first started with a simple with open on a file, which allowed me to use code
that "iter"'s.
for example:
for meet in root.iter("meeting"):
for race in root.iter("race"):
for nom in root.iter("nomination"):
meetattr = meet.attrib
I then got correct output so wanted to scale up to passing a directory of
files. So skipping the code that deals with getting it off the command line I
implemented a generator to yield the file root object as needed.
This is that code
def return_files(file_list):
"""
Take a list of files and return file when called.
Calling function to supply attributes
"""
for filename in sorted(file_list):
with open(dir_path + filename) as fd:
tree = etree.parse(fd)
root = tree.getroot()
yield root
My question is though now that I have implemented it this way my
I pull in the root via a function first few lines are
def dataAttr(roots):
"""Get the root object and iter items."""
with open("output/first2.csv", 'w', newline='') as csvf:
race_writer = csv.writer(csvf, delimiter=',')
for meet in roots.iter("meeting"):
which I call as
rootObs = return_files(file_list)
dataAttr(rootObs)
So if I use a generator to pass in the root lxml object to a function how do I
iter since python provides an error that iters don't exist on python objects?
This is said error
± |master U:1 ?:1 ✗| → python3 race.py data/ -e *.xml
Traceback (most recent call last):
File "race.py", line 77, in
dataAttr(rootObs)
File "race.py", line 55, in dataAttr
for meet in roots.iter("meeting"):
AttributeError: 'generator' object has no attribute 'iter'
Cheers
Sayth
--
https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On 10/01/2016 05:21 AM, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the > latest Python 3.6, and no time to upgrade my usual machines to > something which will work. > > However I do have access to another machine (actually a VM) which > can compile Python 3.6. It's not practical for me to use it as a my > main development machine, but as a temporary measure, I thought I > could compile 3.6 on this VM, then copy the python binary to my usual > desktop machine. > > What sort of challenges am I likely to find? Both machines are Linux, > but different distros. If they use the same major version of glibc, both are either 64-bit or 32-bit, then if you compile to a self-contained location like /opt and then copy that folder over it will run just fine. If you compile to /usr, you'll have to make sure you tar up all the bits that are scattered around the various directories under /usr. What distros are we talking about here? Ideally it would be nice to build install-able packages for the target OS. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
I'm not sure any more to what message this should be a followup, but
here is a demonstration of two different semantics of the for-loop
variable scope/update, this time with nested loops using the same loop
variable name. The first function, tabulate, uses Python semantics ("t"
for true, if you like); the second, fabulate, is a translation ("f" for
false, if you like) that uses the magical semantics where the loop
variable is not only local to the loop but also a different variable on
each iteration. The latter property makes no difference in this
demonstration, but the former does; there's also a spurious counter that
is not local to the nested loops, just to be sure that it works as
expected (it does).
A summary of sorts: it's possible to demonstrate the scope difference in
Python code, with no box in sight; boxes are irrelevant; the relevant
issue is what function and when the loop variable is associated with,
explicitly or implicitly.
def tabulate(m, n):
for i in range(m):
print(i, end = ': ')
c = 0
for i in range(n):
print(i, end = ', ' if i + 1 < n else ' : ')
c += 1
print(i, c)
def fabulate(m, n):
c = None # because c belong in this scope
g1 = iter(range(m))
try:
while True:
def g2(i):
nonlocal c
print(i, end = ': ')
c = 0
g3 = iter(range(n))
try:
while True:
def g4(i):
nonlocal c
print(i, end = ', ' if i + 1 < n else ' : ')
c += 1
g4(next(g3))
except StopIteration:
pass
print(i, c)
g2(next(g1))
except StopIteration:
pass
print('Python:')
tabulate(3, 4)
print()
print('Magick:')
fabulate(3, 4)
# Output from the above, each line being
# outer i: each inner i : i after inner loop, c
# where either c correctly counts inner steps but
# Python inner i clobbers outer i, Magick not.
#
# Python:
# 0: 0, 1, 2, 3 : 3 4
# 1: 0, 1, 2, 3 : 3 4
# 2: 0, 1, 2, 3 : 3 4
#
# Magick:
# 0: 0, 1, 2, 3 : 0 4
# 1: 0, 1, 2, 3 : 1 4
# 2: 0, 1, 2, 3 : 2 4
--
https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Oct 1, 2016 06:25, "Steve D'Aprano" wrote: > > Long story short: I have no working systems capable of compiling the latest > Python 3.6, and no time to upgrade my usual machines to something which > will work. Since you're working on a pure-Python module (statistics), I'd recommend updating to the latest changeset that will build, and work from there. I don't know of any changes that would make it impossible for you to update to the last working changeset, build, update to 3.6 tip, and develop and test your statistics changes without rebuilding. You may have some test failures, just run the full test suite after update to determine which failures are not due to your development. Otherwise, I'd suggest compiling your own GCC, but I haven't tried that myself without the aid of Gentoo's package manager. -- Zach (On a phone) -- https://mail.python.org/mailman/listinfo/python-list
ConfigParser: use newline in INI file
Hi, ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to signal to ConfigParser that there is a line break? Thorsten -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Saturday, October 1, 2016 at 7:02:58 PM UTC+5:30, Jussi Piitulainen wrote:
> Rustom Mody writes:
>
> > And then define comprehensions not as now done in terms of for loops
> > that mutatingly extend the list being built up but as recursive
> > functions that get (re)called for every new value of the comprehension
> > variable passed and therefore fresh-bound as parameter
>
> You'd get the magical semantics for comprehensions from the current
> definition of comprehension semantics in terms of the loop, if the loop
> semantics was magical. Let me demonstrate. (The b-word is unfortunately
> not available, so I substitute "magical" instead.)
>
> # delayd = [ lambda : c for c in "abracadabra" ]
> # is roughly/almost equal to the following.
>
> delayd = []
> for c in "abracadabra":
> delayd.append(lambda : c)
>
> print('Python:', ''.join(f() for f in delayd))
>
> # But that for-loop could have been roughly equal to the following,
> # giving both the comprehension and the underlying for-loop a
> # semantics that some people say they would prefer.
>
> delayd = [] # reset the list, not part of the loop
>
> g1 = iter("abracadabra")
> try:
> while True:
> def g2(c):
> delayd.append(lambda : c)
> g2(next(g1))
> except StopIteration:
> pass
>
> print('Magick:', ''.join(f() for f in delayd))
>
> # Output from the above:
> # Python: aaa
> # Magick: abracadabra
Hoo boy1
Thats some tour de force and makes my head spin
Point can be made more simply with map
ie if we *define*
[exp for cv in l]
as
map(lambda cv: exp, l)
the problem vanishes
Demo:
First a helper function for demoing:
def pam(fl,x):
return map(lambda f: f(x), fl)
# pam is the complement to map; map runs one fnc on a list of args
# pam runs a list of funcs on one arg
Trying to make a list of functions that add one, two and three to their
arguments
fl = [lambda x: x + cv for cv in [1,2,3]]
Broken because of python's wrong LC semantics:
>>> pam(fl, 3)
[6, 6, 6]
Transform the LC into a map with the rule above:
fl_good = map((lambda cv :lambda x: x+cv), [1,2,3])
Works!
>>> pam(fl_good, 3)
[4, 5, 6]
>>>
Which is not very far from the standard workaround for this gotcha:
>>> fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]]
>>> pam(fl_workaround, 3)
[4, 5, 6]
>>>
Maybe we could say the workaround is the map definition uncurried
And then re-comprehension-ified
--
https://mail.python.org/mailman/listinfo/python-list
RASTER analysis(slope)
hello team i want to calculate slope and aspect from some RASTER IMAGE(.grid,tiff,geotiff) who is the better method to can i do this ? with numpy and scipy or with some package? thnx you team -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sun, 2 Oct 2016 03:57 am, Rustom Mody wrote: > Hoo boy1 > Thats some tour de force and makes my head spin I certainly agree with the second part of your sentence. > Point can be made more simply with map > ie if we *define* > [exp for cv in l] > as > map(lambda cv: exp, l) > > the problem vanishes > > Demo: > > First a helper function for demoing: > > def pam(fl,x): > return map(lambda f: f(x), fl) > # pam is the complement to map; map runs one fnc on a list of args > # pam runs a list of funcs on one arg > > Trying to make a list of functions that add one, two and three to their > arguments > > fl = [lambda x: x + cv for cv in [1,2,3]] > > Broken because of python's wrong LC semantics: pam(fl, 3) > [6, 6, 6] Its not *broken*, its doing *exactly what you told it to do*. You said, define a function that takes a single argument x, and return x + cv. Then you delayed evaluating it until cv = 3, and passed the argument 3, so of course it returns 6. That's exactly what you told it to calculate. You seem to have the concept that lambda should be magical, and just miraculously know how far back in time to look for the value of cv. And then when it doesn't, you're angry that Python is "broken". But why should it be magical? cv is just an ordinary variable, and like all variables, looking it up returns the value it has at the time you do the look-up, not some time in the past. Let's unroll the loop: fl = [] cv = 1 def f(x): return x + cv fl.append(f) cv = 2 def f(x): return x + cv fl.append(f) cv = 3 def f(x): return x + cv fl.append(f) pam(fl, 3) Are you still surprised that it returns [6, 6, 6]? > Transform the LC into a map with the rule above: > fl_good = map((lambda cv :lambda x: x+cv), [1,2,3]) This is equivalent to something completely different, using a closure over cv, so of course it works: def factory(cv): def inner(x): return x + cv return inner fl_good = [] fl_good.append(factory(1)) fl_good.append(factory(2)) fl_good.append(factory(3)) Each time you call factory(), you get a new scope, with its own independent variable cv. The inner function captures that environment (a closure), which includes that local variable cv. Each invocation of factory leads to an inner function that sees a different local variable which is independent of the others but happens to have the same name. Instead of three functions all looking up a single cv variable, you have three functions looking up three different cv variables. This is essentially why closures exist. > Which is not very far from the standard workaround for this gotcha: fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]] pam(fl_workaround, 3) > [4, 5, 6] > > Maybe we could say the workaround is the map definition uncurried > And then re-comprehension-ified If your students think in terms of map, then fine, but I think it would confuse more people than it would help. Your mileage may vary. There are certainly a number of ways to get the desired behaviour. If you prefer to work with map, go right ahead. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: ConfigParser: use newline in INI file
On 10/1/2016 10:56 AM, Thorsten Kampe wrote: ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to signal to ConfigParser that there is a line break? Without an example or two, I don't really understand the question enough to answer. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: rocket simulation game with just using tkinter
On 1-10-2016 15:44, Christian Gollwitzer wrote: > Am 01.10.16 um 00:59 schrieb Irmen de Jong: >> Hi, >> >> I've made a very simple rocket simulation game, inspired by the recent >> success of SpaceX > >> You can get the code here if you want to give it a try: >> https://github.com/irmen/rocketsimulator > > Nice! I'll have to rebind the keys before I can successfully play this, > though. My [] > "keys" are in fact combinations of Alt+5 and Alt+6 on a German Macbook > keyboard (on a > German PC, it is AltGr+8, AltGr+9). Why not using the arrow keys? These > should be pretty > universal Oh, I'm sorry about that, my little knowledge of non-US keyboard layouts shows. Arrow keys could be an option. For my education what are the 2 keys to the right of the P then on your keyboard? Rebinding the keys should be easy though just change them in the keypress and keyrelease methods. Irmen -- https://mail.python.org/mailman/listinfo/python-list
Re: ConfigParser: use newline in INI file
Thorsten Kampe writes: > ConfigParser escapes `\n` in ini values as `\\n`. How do you demonstrate that? Here is an example text of a config file:: >>> import io >>> import textwrap >>> config_text = textwrap.dedent(r""" ... [foo] ... wibble = Lorem\nipsum. ... """) >>> print(config_text) [foo] wibble = Lorem\nipsum. So that text has the characters “\n” in it. (Note that I had to use the ‘r’ prefix on the string literal, in order to turn off the special meaning of “\” and get that character literally.) When I use a ConfigParser it reads “\n” and reproduces exactly those characters:: >>> import configparser >>> config_file = io.StringIO(config_text) >>> config = configparser.ConfigParser() >>> config.read_file(config_file) >>> print(config['foo']['wibble']) Lorem\nipsum. So you see that the “\n” characters are preserved exactly by the ConfigParser. > Is there a way to signal to ConfigParser that there is a line break? Yes, you use a line break. See the ‘configparser’ module documentation: Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser’s mode, blank lines may be treated as parts of multiline values or ignored. https://docs.python.org/3/library/configparser.html#supported-ini-file-structure> Thus:: >>> config_text = textwrap.dedent(r""" ... [foo] ... wibble = Lorem ... ipsum. ... """) >>> print(config_text) [foo] wibble = Lorem ipsum. >>> config_file = io.StringIO(config_text) >>> config = configparser.ConfigParser() >>> config.read_file(config_file) >>> print(config['foo']['wibble']) Lorem ipsum. -- \ “Only the shallow know themselves.” —Oscar Wilde, _Phrases and | `\ Philosophies for the Use of the Young_, 1894 | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On 10/1/2016 8:24 AM, Rustom Mody wrote: Yeah by comprehension-variable I mean the one that sits left of the ‘in’ inside the conprehension. In other words, a 'loop variable within a comprehension'. Keep in mind that there may be multiple targets for the implicit (hidden) assignment, so there may be multiple loop (comprehension) variables even without nested loops. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: ConfigParser: use newline in INI file
* Terry Reedy (Sat, 1 Oct 2016 15:44:39 -0400)
>
> On 10/1/2016 10:56 AM, Thorsten Kampe wrote:
>
> > ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to
> > signal to ConfigParser that there is a line break?
>
> Without an example or two, I don't really understand the question enough
> to answer.
>>> !cat INI.ini
[Asciidoc]
_test_stdout = \nHello, World!\n
>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read('INI.ini')
['INI.ini']
>>> config['Asciidoc']['_test_stdout']
'\\nHello, World!\\n'
...as you can see, ConfigParser escaped the backslash by doubling it.
Which is fine in most cases - except when I want to have something
indicating an newline.
Thorsten
--
https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On Sat, Oct 1, 2016 at 11:41 PM, wrote: > Fython permits to write numerical code with the same syntax then Python. > Under the hood, the code is transpiled to Fortran and run at top speed. How does this compare to Python+Numpy? How much faster is Fython, and what are the restrictions on the Python code? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sat, Oct 1, 2016 at 11:03 PM, Steve D'Aprano wrote: >> Are both Linuxes of broadly similar vintage? > > That depends on what you mean by "broadly similar". As far as I am > concerned, a five year difference is not very much, and is broadly > similar -- it's not like I'm using Linux from 1991. But the whole point is > that I need something with gcc 4.8 (maybe 4.7 will do, not sure) but > certainly not 4.4 which is the most recent I can get on my current systems. It's a rubbery term, but basically take the year of release of the distro versions you're using, push 'em forward if they're bleeding-edge distros or back if they're uber-stable, and try to guess the age of the libraries they're running. Or check by looking at some nice over-arching version number like Linux kernel; if they use similar versions of Linux and libc, they probably have roughly similar versions of a lot of other libraries, too. The more similar the library versions, the less issues you'll have. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: ConfigParser: use newline in INI file
* Ben Finney (Sun, 02 Oct 2016 07:12:46 +1100) > > Thorsten Kampe writes: > > > ConfigParser escapes `\n` in ini values as `\\n`. Indenting solves the problem. I'd rather keep it one line per value but it solves the problem. Thorsten -- https://mail.python.org/mailman/listinfo/python-list
Re: RASTER analysis(slope)
On Sunday, 2 October 2016 04:52:13 UTC+11, Xristos Xristoou wrote: > hello team > > i want to calculate slope and aspect from some RASTER > IMAGE(.grid,tiff,geotiff) > who is the better method to can i do this ? > with numpy and scipy or with some package? > > > thnx you team I don't know much about the topic however pyDem seems like the appropriate library to use. https://pypi.python.org/pypi/pyDEM/0.1.1 Here is a standford article on it https://pangea.stanford.edu/~samuelj/musings/dems-in-python-pt-3-slope-and-hillshades-.html and a pdf from scipy conference. https://conference.scipy.org/proceedings/scipy2015/pdfs/mattheus_ueckermann.pdf Sayth -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
Steve D'Aprano writes: > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could > compile 3.6 on this VM, then copy the python binary to my usual > desktop machine. How about installing the same OS on the VM that you're running on the development machine? You can get powerful hourly VM's (x86) and dedicated servers (x86 and ARM) really cheap at Scaleway.com. I've used them for a while and they work well. -- https://mail.python.org/mailman/listinfo/python-list
Byte code descriptions somewhere?
Hi all, I've all of a sudden gotten interested in the CPython interpreter, and started trying to understand how it ingests and runs byte code. I found Include/opcode.h in the python sources, and I found some basic documentation on how to add in new opcodes online, but I haven't found the equivalent of an assembly manual like you might for x86, etc. Is there something similar to a manual dedicated to python byte code? Also, is there a manual for how the interpreter expects the stack, etc. to be setup so that all interactions go as expected (garbage collections works, exceptions work, etc.)? Basically, I want a manual similar to what Intel or AMD might put out for their chips so that all executables behave nicely with one another. Thanks, Cem Karan -- https://mail.python.org/mailman/listinfo/python-list
Re: Byte code descriptions somewhere?
Cem Karan writes: > Hi all, I've all of a sudden gotten interested in the CPython > interpreter, and started trying to understand how it ingests and runs > byte code. That sounds like fun! > Is there something similar to a manual dedicated to python byte code? The Python documentation for the ‘dis’ module shows not only how to use that module for dis-assembly of Python byte code, but also a reference for the byte code. 32.12. dis — Disassembler for Python bytecode https://docs.python.org/3/library/dis.html> -- \ “Skepticism is the highest duty and blind faith the one | `\ unpardonable sin.” —Thomas Henry Huxley, _Essays on | _o__) Controversial Questions_, 1889 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Byte code descriptions somewhere?
Cool, thank you! Quick experimentation suggests that I don't need to worry about marking anything for garbage collection, correct? The next question is, how do I create a stream of byte codes that can be interpreted by CPython directly? I don't mean 'use the compile module', I mean writing my own byte array with bytes that CPython can directly interpret. Thanks, Cem Karan On Oct 1, 2016, at 7:02 PM, Ben Finney wrote: > Cem Karan writes: > >> Hi all, I've all of a sudden gotten interested in the CPython >> interpreter, and started trying to understand how it ingests and runs >> byte code. > > That sounds like fun! > >> Is there something similar to a manual dedicated to python byte code? > > The Python documentation for the ‘dis’ module shows not only how to use > that module for dis-assembly of Python byte code, but also a reference > for the byte code. > >32.12. dis — Disassembler for Python bytecode > >https://docs.python.org/3/library/dis.html> > > -- > \ “Skepticism is the highest duty and blind faith the one | > `\ unpardonable sin.” —Thomas Henry Huxley, _Essays on | > _o__) Controversial Questions_, 1889 | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Byte code descriptions somewhere?
Cem Karan writes: > how do I create a stream of byte codes that can be interpreted by > CPython directly? Basically, study the already existing code and do something similar. The CPython bytecode isn't standardized like JVM bytecode. It's designed for the interpreter's convenience, not officially documented, and (somewhat) subject to change between versions. -- https://mail.python.org/mailman/listinfo/python-list
Re: Byte code descriptions somewhere?
On Sun, Oct 2, 2016 at 10:47 AM, Cem Karan wrote:
> Cool, thank you! Quick experimentation suggests that I don't need to worry
> about marking anything for garbage collection, correct? The next question
> is, how do I create a stream of byte codes that can be interpreted by CPython
> directly? I don't mean 'use the compile module', I mean writing my own byte
> array with bytes that CPython can directly interpret.
>
"Marking for garbage collection" in CPython is done by refcounts; the
bytecode is at a higher level than that.
>>> dis.dis("x = y*2")
1 0 LOAD_NAME0 (y)
3 LOAD_CONST 0 (2)
6 BINARY_MULTIPLY
7 STORE_NAME 1 (x)
10 LOAD_CONST 1 (None)
13 RETURN_VALUE
A LOAD operation will increase the refcount (a ref is on the stack),
BINARY_MULTIPLY dereferences the multiplicands and adds a ref to the
product, STORE will deref whatever previously was stored, etc.
To execute your own code, look at types.FunctionType and
types.CodeType, particularly the latter's 'codestring' argument
(stored as the co_code attribute). Be careful: you can easily crash
CPython if you mess this stuff up :)
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Converting PyMember_Get usage to PyMember_GetOne - no docs?
I'm trying to convert the python-sybase module C code from oldish Python 2 usage to work in Python 3. The code currently calls PyMember_Get(), which is obsolete. In later versions of Python 2, PyMember_GetOne became the official way to do things, the the former function was still available. In Python 3, PyMember_Get() is gone. This is all well and good, but I can't find documentation for either function, or seemingly examples of code which converted from one to the other. Any pointers? Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On 2016-10-01, Chris Angelico wrote: > On Sat, Oct 1, 2016 at 11:41 PM, wrote: >> Fython permits to write numerical code with the same syntax then Python. >> Under the hood, the code is transpiled to Fortran and run at top speed. > > How does this compare to Python+Numpy? How much faster is Fython, and > what are the restrictions on the Python code? > > ChrisA Interested to hear the answers to those questions, and whether Fython is pass by reference (Fortran) or value (python, unless passing a list, for example); and then there's the 'little' matter of one-based (Fortran) or zero-based (python) arrays? -- https://mail.python.org/mailman/listinfo/python-list
inplace text filter - without writing file
Hi
I have a fileobject which was fine however now I want to delete a line from the
file object before yielding.
def return_files(file_list):
for filename in sorted(file_list):
with open(dir_path + filename) as fd:
for fileItem in fd:
yield fileItem
Ned gave an answer over here http://stackoverflow.com/a/6985814/461887
for i, line in enumerate(input_file):
if i == 0 or not line.startswith('#'):
output.write(line)
which I would change because it is the first line and I want to rid
Re: Byte code descriptions somewhere?
On Saturday, October 1, 2016 at 7:48:09 PM UTC-4, Cem Karan wrote: > Cool, thank you! Quick experimentation suggests that I don't need to worry > about marking anything for garbage collection, correct? The next question > is, how do I create a stream of byte codes that can be interpreted by CPython > directly? I don't mean 'use the compile module', I mean writing my own byte > array with bytes that CPython can directly interpret. In Python 2, you use new.code: https://docs.python.org/2/library/new.html#new.code It takes a bytestring of byte codes as one of its twelve (!) arguments. Something that might help (indirectly) with understanding bytecode: byterun (https://github.com/nedbat/byterun) is a pure-Python implementation of a Python bytecode VM. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On Sun, Oct 2, 2016 at 11:18 AM, Raoul Fleckman wrote: > Interested to hear the answers to those questions, and whether Fython is > pass by reference (Fortran) or value (python, unless passing a list, for > example); and then there's the 'little' matter of one-based (Fortran) or > zero-based (python) arrays? Python doesn't have two different pass-by-X conventions. Please read and comprehend: http://nedbatchelder.com/text/names1.html ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: ConfigParser: use newline in INI file
On Saturday, October 1, 2016 at 6:25:16 PM UTC-4, Thorsten Kampe wrote:
> * Ben Finney (Sun, 02 Oct 2016 07:12:46 +1100)
> >
> > Thorsten Kampe writes:
> >
> > > ConfigParser escapes `\n` in ini values as `\\n`.
>
> Indenting solves the problem. I'd rather keep it one line per value
> but it solves the problem.
If you want to have \n mean a newline in your config file, you can
do the conversion after you read the value:
>>> "a\\nb".decode("string-escape")
'a\nb'
--Ned.
--
https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Steve D'Aprano wrote: When you say: x = 0 x = 1 inside a function, and the interpreter does the name binding twice, there's no way of telling whether it writes to the same cell each time or not. Yes, there is: ... x = 0 ... f1 = lambda: x ... x = 1 ... f2 = lambda: x ... print(f1(), f2()) ... >>> f() 1 1 This indicates that both assignments updated the same slot. Otherwise the result would have been "0 1". It's not currently possible to observe the other behaviour in Python, because the only way to create new bindings for local names is to enter a function. The change to for-loop semantics I'm talking about would introduce another way. Certainly when you call a function, the local bindings need to be created. Obviously they didn't exist prior to calling the function! I didn't think that was the difference you were referring to, and I fail to see how it could be relevant to the question of for-loop behaviour. My proposed change is (mostly) equivalent to turning the loop body into a thunk and passing the loop variable in as a parameter. This is the way for-loops or their equivalent are actually implemented in Scheme, Ruby, Smalltalk and many other similar languages, which is why they don't have the same "gotcha". (Incidentally, this is why some people describe Python's behaviour here as "broken". They ask -- it works perfectly well in these other languages, why is Python different?) The trick with cells is a way to get the same effect in CPython, without the overhead of an actual function call on each iteration (and avoiding some of the other problems that using a thunk would entail). The cell trick isn't strictly necessary, though -- other Python implementations could use a thunk if they had to. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: generator no iter - how do I call it from another function
Sayth Renshaw wrote:
def dataAttr(roots):
"""Get the root object and iter items."""
with open("output/first2.csv", 'w', newline='') as csvf:
race_writer = csv.writer(csvf, delimiter=',')
for meet in roots.iter("meeting"):
The value of roots you're passing in here is a generator
object that yields root instances, not a root instance itself.
So it looks like you need another level of iteration:
for root in roots:
for meet in root.iter("meeting"):
...
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list
Re: Lawrence D'Oliveiro
On Fri, 30 Sep 2016 23:29:41 -0700, Paul Rubin declaimed the following: That's the weird Italian spam that the newsgroup has been getting for a while. I've been wondering for a while if anyone knows what the story is, i.e. why it's on comp.lang.python but not on other newsgroups that I've noticed. The Spanish Inquisition is trying to get the Pope impeached, or something? -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: inplace text filter - without writing file
On 2016-10-02 01:21, Sayth Renshaw wrote:
Hi
I have a fileobject which was fine however now I want to delete a line from the
file object before yielding.
def return_files(file_list):
for filename in sorted(file_list):
When joining paths together, it's better to use 'os.path.join'.
with open(dir_path + filename) as fd:
for fileItem in fd:
yield fileItem
Ned gave an answer over here http://stackoverflow.com/a/6985814/461887
for i, line in enumerate(input_file):
if i == 0 or not line.startswith('#'):
output.write(line)
which I would change because it is the first line and I want to rid
Need help for the print() function with a better order
I am trying to print a simple decision tree for my homework.
The answer must keep in this format:
Top 7,4,0.95
career gain = 100
1.Management 2, 3, 0.9709505944546686
2.Service 5, 1, 0.6500224216483541
location gain = 100
1.Oregon 4, 1, 0.7219280948873623
2.California 3, 3, 1.0
edu_level gain = 100
1.High School 5, 1, 0.6500224216483541
2.College 2, 3, 0.9709505944546686
years_exp gain = 100
1.Less than 3 3, 1, 0.8112781244591328
2.3 to 10 2, 1, 0.9182958340544896
3.More than 10 2, 2, 1.0
Here is my code:
features={'edu_level':['High School','College'],'career':
['Management','Service'],'years_exp':['Less than 3','3 to 10','More than
10'],'location':['Oregon','California']}
print('Top 7,4,0.95')
for key in features:
print('{} gain = {}'.format(key,100))
attributes_list=features[key]
kargs={}
for i in range(len(attributes_list)):
kargs[key]=attributes_list[i]
low=table.count('Low',**kargs)
high=table.count('High',**kargs)
print('\t{}.{} {}, {},
{}'.format(i+1,attributes_list[i],low,high,entropy(low,high)))
I set all the gain as 100 now.But actually the gain must calculate with the
data below.
For example, the career gain need the data of 'Management' and 'Service'.
I don't know how to do.
or Anyone can provide me a better logic?
--
https://mail.python.org/mailman/listinfo/python-list
Thonny 2.0 released (Python IDE for beginners)
Hi! Thonny is Python IDE for learning and teaching programming. It is developed in University of Tartu, Estonia. It has an easy to use debugger which shows clearly how Python executes your programs. Unlike most debuggers, it can even show the steps of evaluating an expression, visually explain references, function calls, exceptions etc. For more info and downloads see http://thonny.cs.ut.ee/ [1] best regards, Aivar Annamaa University of Tartu Institute of Computer Science http://thonny.cs.ut.ee";>Thonny 2.0 - Python IDE for beginners (01-Oct-16) Links: -- [1] http://thonny.cs.ut.ee/ -- https://mail.python.org/mailman/listinfo/python-list
Merecurial and Python-2.7.x, Python-3.Y
Finally, I got to where I understood what needed to be done to get both Mercurial built - and the the new SSL requirements met. So, running: # hg clone https://hg.python.org/cpython works. What is the next step to getting Python-2.7 AND Python-3.7 so I can submit patches against both versions and/or other versions? Is it going to be a command like in the developer guide (that references Python3.5)? Does this create a new directory, or just undo a lot of things that was just cloned - or was 3.5 the development branch when the guide was last updated? Will I need to clone (as above) several times - one for each version I want to test against - and then run a second command -- OR -- is there an hg clone command to get a specific version (If that is in the guide - my apologies, as I missed it. Thanks for hints and patience! M -- https://mail.python.org/mailman/listinfo/python-list
Re: Byte code descriptions somewhere?
On Saturday, October 1, 2016 at 11:57:17 PM UTC+1, Cem Karan wrote: > Hi all, I've all of a sudden gotten interested in the CPython interpreter, > and started trying to understand how it ingests and runs byte code. I found > Include/opcode.h in the python sources, and I found some basic documentation > on how to add in new opcodes online, but I haven't found the equivalent of an > assembly manual like you might for x86, etc. Is there something similar to a > manual dedicated to python byte code? Also, is there a manual for how the > interpreter expects the stack, etc. to be setup so that all interactions go > as expected (garbage collections works, exceptions work, etc.)? Basically, I > want a manual similar to what Intel or AMD might put out for their chips so > that all executables behave nicely with one another. > > Thanks, > Cem Karan Further to Ben Finney's answer this https://docs.python.org/devguide/compiler.html should help. Kindest regards. Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list
Reading the Windows Location API
I found some GeoLocation stuff on PyPi, but it seems to be focused on turning an IP address into a (rough) location rather than reading a GPS. Seems like if a GPS is attached, reading it would be the best way to obtain a more accurate location, falling back to approximations if there is no GPS. But I'm not sure where to start with reading the Windows Location API. Anyone have some clues or sample code? -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sun, 2 Oct 2016 09:31 am, Paul Rubin wrote: > Steve D'Aprano writes: >> However I do have access to another machine (actually a VM) which can >> compile Python 3.6. It's not practical for me to use it as a my main >> development machine, but as a temporary measure, I thought I could >> compile 3.6 on this VM, then copy the python binary to my usual >> desktop machine. > > How about installing the same OS on the VM that you're running on the > development machine? Because then I won't be able to build Python on the VM either, which completely misses the point. I already have three machines that cannot build Python. I don't need a fourth. I need one which *can* build Python. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On Sun, 2 Oct 2016 11:18 am, Raoul Fleckman wrote: > Interested to hear the answers to those questions, and whether Fython is > pass by reference (Fortran) or value (python, unless passing a list, for > example); That's not how Python works. Ints and lists are passed exactly the same way, neither by value nor by reference. http://import-that.dreamwidth.org/1130.html -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
**How does this compare to Python+Numpy? **How much faster is Fython, and what are the restrictions on the Python code? Python+Numpy allows easy processing of vector, but there is a limit to how much user-defined logic can be used with Numpy. For example, operating on three different arrays to do something like if x[i] > 0 : temp = get_value( y[i] ) if temp > 0: z[i] = another_operation( z[i] ) may require three different Numpy calls. With Fython, the if statements and the loop around them is directly compiled to machine code. Fython speed is the same as Fortran speed. In some applications, I get a speed-up of 100 compared to Cython, and a speed-up of 10 compared to C. I'm not a Cython nor a C expert, and I know that with some good compiler flags, one may make the performance of these two languages similar two Fortran. The advantage of Fython is that everything is tune out-of-box because of its connection to Fortran. There is no restriction on the Python code. Once a Fython program is avalaible, Python can throw any scalar or Numpy array at it. **whether Fython is pass by reference (Fortran) or value **and then there's the 'little' matter of one-based (Fortran) or zero-based (python) arrays? Fython is passed by reference, so that any modification made to a variable in Fython is propagated back to Python. Fython follows Fortran syntax and is one-based by default. This doesn't give any interop problem with Python, because what matters is to know the pointer to the first element of an array, and Fython knows it. For the programmer convenience, like Fortran, Fython allows the choice between one-based and zero-based indexing at array declaration. -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sun, Oct 2, 2016 at 1:51 PM, Steve D'Aprano wrote: > On Sun, 2 Oct 2016 09:31 am, Paul Rubin wrote: > >> Steve D'Aprano writes: >>> However I do have access to another machine (actually a VM) which can >>> compile Python 3.6. It's not practical for me to use it as a my main >>> development machine, but as a temporary measure, I thought I could >>> compile 3.6 on this VM, then copy the python binary to my usual >>> desktop machine. >> >> How about installing the same OS on the VM that you're running on the >> development machine? > > Because then I won't be able to build Python on the VM either, which > completely misses the point. I already have three machines that cannot > build Python. I don't need a fourth. I need one which *can* build Python. Hmm, I've possibly missed something here, which may indicate a problem. Why can't your existing machines build? Is it because they have too-old versions of tools, and if so, which? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On Sun, Oct 2, 2016 at 1:58 PM, wrote: > Fython speed is the same as Fortran speed. > > There is no restriction on the Python code. > Once a Fython program is avalaible, Python can throw any scalar or Numpy > array at it. > > For the programmer convenience, like Fortran, Fython allows the choice between > one-based and zero-based indexing at array declaration. This is where I'm a little confused. Is Fython a full-featured Python implementation that compiles to Fortran, or a thin wrapper around Fortran that uses Python syntax, or something in between? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On 10/01/2016 09:06 PM, Chris Angelico wrote:
> On Sun, Oct 2, 2016 at 1:58 PM, wrote:
>> Fython speed is the same as Fortran speed.
>>
>> There is no restriction on the Python code.
>> Once a Fython program is avalaible, Python can throw any scalar or Numpy
>> array at it.
>>
>> For the programmer convenience, like Fortran, Fython allows the choice
>> between
>> one-based and zero-based indexing at array declaration.
>
> This is where I'm a little confused. Is Fython a full-featured Python
> implementation that compiles to Fortran, or a thin wrapper around
> Fortran that uses Python syntax, or something in between?
>From the web page, it's appears to be just Fortran with a Python-ish
syntax, but with a special Python module ("import fython") to more
easily allow the use of Fortran or Fython constructs (compiled fortran
code) from within actual Python. Seems to allow the blending of
Fortran, Python, and Numpy. Sounds pretty useful. A transpiler and
also a Python bridge.
--
https://mail.python.org/mailman/listinfo/python-list
Re: inplace text filter - without writing file
Thank you -- https://mail.python.org/mailman/listinfo/python-list
Re: announcing fython
On 2016-10-02, nicolasessisbre...@gmail.com : > **How does this compare to Python+Numpy? > **How much faster is Fython, and what are the restrictions on the > **Python code? > > Python+Numpy allows easy processing of vector, but there is a limit to > how much user-defined logic can be used with Numpy. For example, > operating on three different arrays to do something like > > if x[i] > 0 : > temp = get_value( y[i] ) > > if temp > 0: > z[i] = another_operation( z[i] ) > > may require three different Numpy calls. > With Fython, the if statements and the loop around them is directly > compiled to machine code. > Fython speed is the same as Fortran speed. > > In some applications, I get a speed-up of 100 compared to Cython, and > a speed-up of 10 compared to C. > > I'm not a Cython nor a C expert, and I know that with some good > compiler flags, one may make the performance of these two languages > similar two Fortran. The advantage of Fython is that everything is > tune out-of-box because of its connection to Fortran. > > There is no restriction on the Python code. > Once a Fython program is avalaible, Python can throw any scalar or > Numpy array at it. > > **whether Fython is pass by reference (Fortran) or value > **and then there's the 'little' matter of one-based (Fortran) or > **zero-based (python) arrays? > > Fython is passed by reference, so that any modification made to a > variable in Fython is propagated back to Python. > > Fython follows Fortran syntax and is one-based by default. > This doesn't give any interop problem with Python, because what > matters is to know the pointer to the first element of an array, and > Fython knows it. > > For the programmer convenience, like Fortran, Fython allows the choice > between one-based and zero-based indexing at array declaration. Thank you! it sounds very interesting indeed, particularly for those of us with a substantial Fortran code base which needs to be updated. Please keep us posted on your progress! -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sun, 2 Oct 2016 12:28 am, Jussi Piitulainen wrote:
> I'm not sure any more to what message this should be a followup, but
> here is a demonstration of two different semantics of the for-loop
> variable scope/update, this time with nested loops using the same loop
> variable name. The first function, tabulate, uses Python semantics ("t"
> for true, if you like); the second, fabulate, is a translation ("f" for
> false, if you like) that uses the magical semantics where the loop
> variable is not only local to the loop but also a different variable on
> each iteration.
I'm sorry, I don't understand what you mean by "Python semantics" versus "a
translation". A translation of what? In what way is it "magical semantics"?
I see nothing magical in your code: it is Python code.
Of course, it is complex, complicated, convoluted, obfuscated, hard to
understand, non-idiomatic Python code, but there's nothing magical in it
(unless you count nonlocal as magic). And it does nothing that can't be
done more simply: just change the tabulate inner loop variable to a
different name.
def fabulate2(m, n):
# The simple, Pythonic, non-complicated way.
for i in range(m):
print(i, end = ': ')
c = 0
for j in range(n):
print(j, end = ', ' if j + 1 < n else ' : ')
c += 1
print(i, c)
Your version of tabulate and fabulate:
py> tabulate(3, 4)
0: 0, 1, 2, 3 : 3 4
1: 0, 1, 2, 3 : 3 4
2: 0, 1, 2, 3 : 3 4
py> fabulate(3, 4)
0: 0, 1, 2, 3 : 0 4
1: 0, 1, 2, 3 : 1 4
2: 0, 1, 2, 3 : 2 4
My simple version of fabulate:
py> fabulate2(3, 4)
0: 0, 1, 2, 3 : 0 4
1: 0, 1, 2, 3 : 1 4
2: 0, 1, 2, 3 : 2 4
> The latter property makes no difference in this
> demonstration, but the former does; there's also a spurious counter that
> is not local to the nested loops, just to be sure that it works as
> expected (it does).
>
> A summary of sorts: it's possible to demonstrate the scope difference in
> Python code, with no box in sight; boxes are irrelevant; the relevant
> issue is what function and when the loop variable is associated with,
> explicitly or implicitly.
I don't know what "scope difference" you think you are demonstrating.
tabulate() has a single scope, fabulate() has multiple scopes because it
has inner functions that take i as argument, making them local to the inner
functions. Um, yeah, of course they are different. They're different
because you've written them differently. What's your point?
As far as I can see, all you have demonstrated is that it is possible to
write obfuscated code in Python. But we already knew that.
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sun, 2 Oct 2016 12:30 am, Zachary Ware wrote: > On Oct 1, 2016 06:25, "Steve D'Aprano" wrote: >> >> Long story short: I have no working systems capable of compiling the > latest >> Python 3.6, and no time to upgrade my usual machines to something which >> will work. > > Since you're working on a pure-Python module (statistics), I'd recommend > updating to the latest changeset that will build, and work from there. I > don't know of any changes that would make it impossible for you to update > to the last working changeset, build, update to 3.6 tip, and develop and > test your statistics changes without rebuilding. That actually sounds like it might be workable. Thanks for the suggestion. Now I just need to work out the latest changeset that works... -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: Copying a compiled Python from one system to another
On Sun, 2 Oct 2016 01:58 pm, Chris Angelico wrote: > Hmm, I've possibly missed something here, which may indicate a > problem. Why can't your existing machines build? Is it because they > have too-old versions of tools, and if so, which? Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most recent any of my systems support is 4.4. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sun, 2 Oct 2016 11:44 am, Gregory Ewing wrote: > Steve D'Aprano wrote: >> When you say: >> >> x = 0 >> x = 1 >> >> inside a function, and the interpreter does the name binding twice, >> there's no way of telling whether it writes to the same cell each time or >> not. > > Yes, there is: > > ... x = 0 > ... f1 = lambda: x > ... x = 1 > ... f2 = lambda: x > ... print(f1(), f2()) > ... > >>> f() > 1 1 > > This indicates that both assignments updated the same slot. > Otherwise the result would have been "0 1". No it doesn't mean that at all. The result you see is compatible with *both* the "update existing slot" behaviour and "create a new slot" behavior. The *easiest* way to prove that is to categorically delete the existing "slot" and re-create it: x = 0 f1 = lambda: x del x assert 'x' not in locals() x = 1 f2 = lambda: x print(f1(), f2()) which will still print exactly the same results. Objection: I predict that you're going to object that despite the `del x` and the assertion, *if* this code is run inside a function, the "x slot" actually does still exist. It's not "really" deleted, the interpreter just makes sure that the high-level behaviour is the same as if it actually were deleted. Well yes, but that's exactly what I'm saying: that's not an objection, it supports my argument! The way CPython handles local variables is an implementation detail. The high-level semantics is *identical* between CPython functions, where local variables live in a static array of "slots" and re-binding always updates an existing slot, and IronPython, where they don't. The only way you can tell them apart is by studying the implementation. In IronPython, you could have the following occur in a function locals, just as it could happen CPython for globals: - delete the name binding "x" - which triggers a dictionary resize - bind a value to x again - because the dictionary is resized, the new "slot" for x is in a completely different position of the dictionary to the old one There is no user-visible difference between these two situations. Your code works the same way whether it is executed inside a function or at the global top level, whether functions use the CPython local variable optimization or not. > It's not currently possible to observe the other behaviour in > Python, because the only way to create new bindings for local > names is to enter a function. Not at all. Delete a name, and the binding for that name is gone. Now assign to that name again, and a new binding must be created, by definition, since a moment ago it no longer existed. x = 1 del x assert 'x' not in locals() x = 2 > The change to for-loop semantics > I'm talking about would introduce another way. I have lost track of how this is supposed to change for-loop semantics. I'm especially confused because you seem to be arguing that by using an implementation which CPython already uses, for-loops would behave differently. So either I'm not reading you right or you're not explaining yourself well. >> Certainly when you call a function, the local bindings need to be >> created. Obviously they didn't exist prior to calling the function! I >> didn't think that was the difference you were referring to, and I fail to >> see how it could be relevant to the question of for-loop behaviour. > > My proposed change is (mostly) equivalent to turning the > loop body into a thunk and passing the loop variable in as > a parameter. A thunk is not really well-defined in Python, because it doesn't exist, and therefore we don't know what properties it will have. But generally when people talk about thunks, they mean something like a light-weight anonymous function without any parameters: https://en.wikipedia.org/wiki/Thunk You say "passing the loop variable in as a parameter" -- this doesn't make sense. Variables are not values in Python. You cannot pass in a *variable* -- you can pass in a name (the string 'x') or the *value* bound to the name, but there's no existing facility in Python to pass in a variable. If there was, you could do the classic "pass by reference" test: Write a *procedure* which takes as argument two variables and swaps their contents but there's no facility to do something like that in Python. So its hard to talk about your hypothetical change except in hand-wavy terms: "Something magically and undefined happens, which somehow gives the result I want." > This is the way for-loops or their equivalent are actually > implemented in Scheme, Ruby, Smalltalk and many other similar > languages, which is why they don't have the same "gotcha". Instead, they presumably have some other gotcha -- "why don't for loops work the same as unrolled loops?", perhaps. In Python, the most obvious gotcha would be that if for-loops introduced their own scope, you would have to declare any other variables in the outer scope nonlocal. So this would work fine as top-level code: x = 1 for i in range(10): print(
Re: Copying a compiled Python from one system to another
On Sun, Oct 2, 2016 at 3:25 PM, Steve D'Aprano wrote: > On Sun, 2 Oct 2016 01:58 pm, Chris Angelico wrote: > >> Hmm, I've possibly missed something here, which may indicate a >> problem. Why can't your existing machines build? Is it because they >> have too-old versions of tools, and if so, which? > > Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most > recent any of my systems support is 4.4. In itself not a problem, but you might find that libc is an incompatibly different version, which would be a pain. But, give it a try. Worst case, it fails with a linker error straight away. (And you MAY be able to just copy in some libraries from the other system.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Sun, Oct 2, 2016 at 3:19 PM, Steve D'Aprano wrote: > In IronPython, you could have the following occur in a function locals, just > as it could happen CPython for globals: > > - delete the name binding "x" > - which triggers a dictionary resize > - bind a value to x again > - because the dictionary is resized, the new "slot" for x is in a > completely different position of the dictionary to the old one > > There is no user-visible difference between these two situations. Your code > works the same way whether it is executed inside a function or at the > global top level, whether functions use the CPython local variable > optimization or not. Hmm, interesting. I don't have IronPython here, but maybe you can tell me what this does: print(str) str = "demo" print(str) del str print(str) and the same inside a function. In CPython, the presence of 'str = "demo"' makes str function-local, ergo UnboundLocalError on the first reference; but globals quietly shadow built-ins, so this will print the class, demo, and the class again. If IronPython locals behave the way CPython globals behave, that would most definitely be a user-visible change to shadowing semantics. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: inplace text filter - without writing file
On Sunday, 2 October 2016 12:14:43 UTC+11, MRAB wrote:
> On 2016-10-02 01:21, Sayth Renshaw wrote:
> > Hi
> >
> > I have a fileobject which was fine however now I want to delete a line from
> > the file object before yielding.
> >
> > def return_files(file_list):
> > for filename in sorted(file_list):
>
> When joining paths together, it's better to use 'os.path.join'.
>
> > with open(dir_path + filename) as fd:
> > for fileItem in fd:
> > yield fileItem
> >
> > Ned gave an answer over here http://stackoverflow.com/a/6985814/461887
> >
> > for i, line in enumerate(input_file):
> > if i == 0 or not line.startswith('#'):
> > output.write(line)
> >
> > which I would change because it is the first line and I want to rid
Re: inplace text filter - without writing file
On Sunday, 2 October 2016 16:19:14 UTC+11, Sayth Renshaw wrote:
> On Sunday, 2 October 2016 12:14:43 UTC+11, MRAB wrote:
> > On 2016-10-02 01:21, Sayth Renshaw wrote:
> > > Hi
> > >
> > > I have a fileobject which was fine however now I want to delete a line
> > > from the file object before yielding.
> > >
> > > def return_files(file_list):
> > > for filename in sorted(file_list):
> >
> > When joining paths together, it's better to use 'os.path.join'.
> >
> > > with open(dir_path + filename) as fd:
> > > for fileItem in fd:
> > > yield fileItem
> > >
> > > Ned gave an answer over here http://stackoverflow.com/a/6985814/461887
> > >
> > > for i, line in enumerate(input_file):
> > > if i == 0 or not line.startswith('#'):
> > > output.write(line)
> > >
> > > which I would change because it is the first line and I want to rid
Re: Copying a compiled Python from one system to another
Steve D'Aprano writes: > Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most > recent any of my systems support is 4.4. Building gcc takes a while but it's reasonably simple. Just start it going and read a book for a while. -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
On Saturday, October 1, 2016 at 11:35:31 PM UTC+5:30, Steve D'Aprano wrote: > On Sun, 2 Oct 2016 03:57 am, Rustom Mody wrote: > > > Hoo boy1 > > Thats some tour de force and makes my head spin > > I certainly agree with the second part of your sentence. > > > > Point can be made more simply with map > > ie if we *define* > > [exp for cv in l] > > as > > map(lambda cv: exp, l) > > > > the problem vanishes > > > > > Demo: > > > > First a helper function for demoing: > > > > def pam(fl,x): > > return map(lambda f: f(x), fl) > > # pam is the complement to map; map runs one fnc on a list of args > > # pam runs a list of funcs on one arg > > > > Trying to make a list of functions that add one, two and three to their > > arguments > > > > fl = [lambda x: x + cv for cv in [1,2,3]] > > > > Broken because of python's wrong LC semantics: > pam(fl, 3) > > [6, 6, 6] > > Its not *broken*, its doing *exactly what you told it to do*. You said, > define a function that takes a single argument x, and return x + cv. Then > you delayed evaluating it until cv = 3, and passed the argument 3, so of > course it returns 6. That's exactly what you told it to calculate. > > You seem to have the concept that lambda should be magical, and just > miraculously know how far back in time to look for the value of cv. And > then when it doesn't, you're angry that Python is "broken". But why should > it be magical? cv is just an ordinary variable, and like all variables, > looking it up returns the value it has at the time you do the look-up, not > some time in the past. Let's unroll the loop: > > fl = [] > cv = 1 > def f(x): return x + cv > fl.append(f) > cv = 2 > def f(x): return x + cv > fl.append(f) > cv = 3 > def f(x): return x + cv > fl.append(f) > > pam(fl, 3) > > Are you still surprised that it returns [6, 6, 6]? > You are explaining the mechanism behind the bug. Thanks. The bug remains. My new car goes in reverse when I put it in first gear but only on full-moon nights with the tank on reserve when the left light is blinking The engineer explains the interesting software bug in the new PCB. Interesting. But the bug remains Likewise here: [2 ** i for i in[1,2]] == [2**1, 2**2] yet this fails [lambda x: x + i for i in [1,2]] == [lambda x:x+1, lambda x:x=2] is a bug for anyone including the OP of this thread > > > > > Transform the LC into a map with the rule above: > > fl_good = map((lambda cv :lambda x: x+cv), [1,2,3]) > > > This is equivalent to something completely different, using a closure over > cv, so of course it works: > > def factory(cv): > def inner(x): > return x + cv > return inner > > fl_good = [] > fl_good.append(factory(1)) > fl_good.append(factory(2)) > fl_good.append(factory(3)) > > > Each time you call factory(), you get a new scope, with its own independent > variable cv. The inner function captures that environment (a closure), > which includes that local variable cv. Each invocation of factory leads to > an inner function that sees a different local variable which is independent > of the others but happens to have the same name. Instead of three functions > all looking up a single cv variable, you have three functions looking up > three different cv variables. > > This is essentially why closures exist. Closures and lambdas are synonymous: martin Fowler's article on lambda has this first sentence: «a programming concept called Lambdas (also called Closures, Anonymous Functions or Blocks)» http://martinfowler.com/bliki/Lambda.html So if you like you can say the bug here is that python lambdas fail to close variables properly. Which is a double-bug because the blame for closure-failure is falling onto the lambda whereas it is the comprehension semantics in terms of loops rather than in terms of closures/cells as Jussi/Greg are trying to show, that is at fault > > > > Which is not very far from the standard workaround for this gotcha: > fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]] > pam(fl_workaround, 3) > > [4, 5, 6] > > > > > Maybe we could say the workaround is the map definition uncurried > > And then re-comprehension-ified > > If your students think in terms of map, then fine, but I think it would > confuse more people than it would help. Your mileage may vary. You are as usual putting words in my mouth. I did not talk of my students or that map is easier than comprehensions I was taking the Haskell comprehension-semantics link (which BTW you posted!) https://www.haskell.org/onlinereport/exps.html#sect3.11 Simplifying it to the simple case of only one for, no ifs This allows the flatmap to become a map and the multiple cases to go Thereby more succinctly highlighting the problem -- https://mail.python.org/mailman/listinfo/python-list
Re: Merecurial and Python-2.7.x, Python-3.Y
On 10/1/2016 5:31 AM, Michael Felt wrote: Finally, I got to where I understood what needed to be done to get both Mercurial built - and the the new SSL requirements met. So, running: # hg clone https://hg.python.org/cpython works. What is the next step to getting Python-2.7 AND Python-3.7 so I can submit patches against both versions and/or other versions? What OS? Is it going to be a command like in the developer guide (that references Python3.5)? I use TortoiseHG on Windows and love it relative to command lines. Does this create a new directory, or just undo a lot of things that was just cloned - or was 3.5 the development branch when the guide was last updated? Will I need to clone (as above) several times - one for each version I want to test against - and then run a second command -- OR -- I recommend one clone from hg.python.org and a share clone for each branch you care about linked to the master clone. Details in the devguide. But I don't know what will have to change when the repository is switched to git and gethup, perhaps next January. is there an hg clone command to get a specific version (If that is in the guide - my apologies, as I missed it. No. By default, the working directory for a clone is for the default branch. Any clone can be 'updated' to any branch. I have each share clone updated to a different branch, so I never update to a different branch. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: unintuitive for-loop behavior
Steve D'Aprano writes:
> On Sun, 2 Oct 2016 12:28 am, Jussi Piitulainen wrote:
>
>> I'm not sure any more to what message this should be a followup, but
>> here is a demonstration of two different semantics of the for-loop
>> variable scope/update, this time with nested loops using the same
>> loop variable name. The first function, tabulate, uses Python
>> semantics ("t" for true, if you like); the second, fabulate, is a
>> translation ("f" for false, if you like) that uses the magical
>> semantics where the loop variable is not only local to the loop but
>> also a different variable on each iteration.
>
> I'm sorry, I don't understand what you mean by "Python semantics" versus "a
> translation". A translation of what? In what way is it "magical semantics"?
> I see nothing magical in your code: it is Python code.
"Python semantics" is the semantics that Python actually uses when it
updated the variables of a for-loop. I would like to call it "assignment
semantics".
The second function is an executable translation of the first function
under a different semantics. I would like to call it "binding semantics"
to distinguish it from "assignment semantics", but that doesn't work
when people insist that "binding" and "assignment" are the same thing in
Python, so I called it "magical semantics" instead.
The method of using equivalent code as an explanation is known in Python
documentation, as in the explanation of list comprehensions in terms of
list.append and for-loops. This is what I called "translation" above. In
another community it might be called "syntactic transformation" or
"macro expansion".
> Of course, it is complex, complicated, convoluted, obfuscated, hard to
> understand, non-idiomatic Python code, but there's nothing magical in
> it (unless you count nonlocal as magic). And it does nothing that
> can't be done more simply: just change the tabulate inner loop
> variable to a different name.
It was stated, in this thread, that it would have been impossible to
make Python for-loops behave the way the corresponding Python code in my
translations of the two nested for-loops behaves. I thought it would be
a good idea to *show* how the "impossible" thing can be done.
I could have just posted that yes, it *could* be done, there's nothing
so special about variables and scope in Python. But I pretty much know
that the response to *that* would have been yet another round of "You
don't understand that Python variables are different, they are not boxes
in fixes locations but name bindings, and no, it cannot be done."
> def fabulate2(m, n):
> # The simple, Pythonic, non-complicated way.
> for i in range(m):
> print(i, end = ': ')
> c = 0
> for j in range(n):
> print(j, end = ', ' if j + 1 < n else ' : ')
> c += 1
> print(i, c)
It misses only the point of the exercise.
> Your version of tabulate and fabulate:
> py> tabulate(3, 4)
> 0: 0, 1, 2, 3 : 3 4
> 1: 0, 1, 2, 3 : 3 4
> 2: 0, 1, 2, 3 : 3 4
>
> py> fabulate(3, 4)
> 0: 0, 1, 2, 3 : 0 4
> 1: 0, 1, 2, 3 : 1 4
> 2: 0, 1, 2, 3 : 2 4
>
> My simple version of fabulate:
>
> py> fabulate2(3, 4)
> 0: 0, 1, 2, 3 : 0 4
> 1: 0, 1, 2, 3 : 1 4
> 2: 0, 1, 2, 3 : 2 4
>
>
>> The latter property makes no difference in this
>> demonstration, but the former does; there's also a spurious counter that
>> is not local to the nested loops, just to be sure that it works as
>> expected (it does).
>>
>> A summary of sorts: it's possible to demonstrate the scope difference in
>> Python code, with no box in sight; boxes are irrelevant; the relevant
>> issue is what function and when the loop variable is associated with,
>> explicitly or implicitly.
>
> I don't know what "scope difference" you think you are demonstrating.
> tabulate() has a single scope, fabulate() has multiple scopes because it
> has inner functions that take i as argument, making them local to the inner
> functions. Um, yeah, of course they are different. They're different
> because you've written them differently. What's your point?
The scope difference is the topic of this thread.
My point is that the for-loops could be translated/compiled/expanded
into the lower-level code so that the loop variables would be in their
own scopes.
> As far as I can see, all you have demonstrated is that it is possible to
> write obfuscated code in Python. But we already knew that.
The point is that the straightforward code *could* have the same meaning
as the "obfuscated" code. That for-loops *could* be magicking constructs
even in Python. (What word should I use instead of "magic"?)
The code might not seem so obfuscated to you if you thought of it not as
source code but as compiled code. Except it's still Python. What word
should I use instead of "translation"? Would "transformation" be
understood?
--
https://mail.python.org/mailman/listinfo/python-list
