Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module

[dl l]

I understood call PyGC_collect to release object cycles. But in my python
test code, seems there is NO reference cycle on the Simple object in the
python test code. Why needs to call PyGC_Collect() here?

=

class Simple:
 def __init__( self ):
 print('Simple__init__')
 def __del__( self ):
 print('Simple__del__')

simple = None

def run():
global simple
simple = Simple()

if __name__ == '__main__':
run()
==


2016-09-21 13:09 GMT+08:00 dieter :

> dl l  writes:
> > I found the problem is resolved if call PyGC_Collect() after
> > PyDict_DelItemString(). Is it expected to call PyGC_Collect() here for
> > Python 3.5 which is not needed for Python 3.3?
>
> Usually, Python uses reference counting to determine when an object
> can be deleted (and then reclaims its memory).
> However, reference counting cannot determine an object's obsoleteness
> when it is involved in a reference cycle. To address most of these
> cases, Python has an alternative way to determine object obsoleteness
> based on marking the accessible objects and garbage collecting
> what is inaccessible. The alternative way is what "PyGC_collect" starts.
>
> If part of the objects where involved in a reference cycle,
> you would have to call "PyGC_collect" also in Python 3.3 in order
> to get a (mostly) clean memory state.
> It might be, that Python 3.5 introduces more cycles than Python 3.3.
> However, more likely, your code was responsible for the cycle creation
> (and not Python itself). Likely, you just did not notice the problem
> back for Python 3.3.
>
> Note that "PyGC_collect", too, is not garanteed to give a memory clean
> state. It does not release object cycles involving objects with
> an explicite destructor ("__del__" method). Those cycles are
> made available to the application (consult the documentation of the
> "gc" module) which can then decide what to do about them
> (e.g. break the cycles and release the associated objects).
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cython taking more time than regular Python

[Gregory Ewing]

Christian Gollwitzer wrote:
However, I'm not convinced it did succeed here. An evaluation of the 
Gauß formula would run in a few *nanoseconds* on any moddern machine. It 
may take a microsecond to call a functino from Python. a hundred 
microseconds means that the loop does run.


Obviously the compiler used was a "dump" compiler. :-)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Data Types

[Steven D'Aprano]

On Wednesday 21 September 2016 14:03, Cai Gengyang wrote:

> So for example for "bool" , it only applies to True/False and nothing else?

Correct. True and False are the only instances of the type 'bool'.

> (2 data types), i.e. :
> 
 type(True)
> 
 type(False)
> 
> 
> Are there any other data types that will give you type(A) or type(B) =  'bool'> besides True and False?

No.





-- 
Steven
git gets easier once you get the basic idea that branches are homeomorphic 
endofunctors mapping submanifolds of a Hilbert space.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module

[dieter]

dl l  writes:
> I found the problem is resolved if call PyGC_Collect() after
> PyDict_DelItemString(). Is it expected to call PyGC_Collect() here for
> Python 3.5 which is not needed for Python 3.3?

Usually, Python uses reference counting to determine when an object
can be deleted (and then reclaims its memory).
However, reference counting cannot determine an object's obsoleteness
when it is involved in a reference cycle. To address most of these
cases, Python has an alternative way to determine object obsoleteness
based on marking the accessible objects and garbage collecting
what is inaccessible. The alternative way is what "PyGC_collect" starts.

If part of the objects where involved in a reference cycle,
you would have to call "PyGC_collect" also in Python 3.3 in order
to get a (mostly) clean memory state.
It might be, that Python 3.5 introduces more cycles than Python 3.3.
However, more likely, your code was responsible for the cycle creation
(and not Python itself). Likely, you just did not notice the problem
back for Python 3.3.

Note that "PyGC_collect", too, is not garanteed to give a memory clean
state. It does not release object cycles involving objects with
an explicite destructor ("__del__" method). Those cycles are
made available to the application (consult the documentation of the
"gc" module) which can then decide what to do about them
(e.g. break the cycles and release the associated objects).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Random832]

On Tue, Sep 20, 2016, at 23:34, Steve D'Aprano wrote:
> One of us is badly missing something.

The problem is the number of times next is called. Here, it'll be more
clear if we wrap the iterator in the original example to print each time
next is called.

class WrappedIterator():
def __init__(self, orig, reference):
self.orig = orig
self.ref = reference
def __iter__(self):
return self
def __next__(self):
desc = "next for " + self.ref
try:
result = next(self.orig)
print(desc, '=', result)
return result
except StopIteration:
print(desc, "StopIteration")
raise

class Node:
def __init__(self, value, left=None, right=None):
self._value = value
self._left = left
self._right = right
def _iter(node):
if node._left:
for x in iter(node._left):
yield x
yield node._value
if node._right:
for x in iter(node._right):
yield x
def __iter__(self):
return WrappedIterator(self._iter(), 'Node %r' % self._value)

node1 = Node(1)
node3 = Node(3)
node2 = Node(2, node1, node3)
node5 = Node(5)
node7 = Node(7)
node6 = Node(6, node5, node7)
node4 = Node(4, node2, node6)

for value in node4:
print("Got value %r" % (value,))

---output---

next for Node 1 = 1
next for Node 2 = 1
next for Node 4 = 1
Got value 1
next for Node 1 StopIteration
next for Node 2 = 2
next for Node 4 = 2
Got value 2
next for Node 3 = 3
next for Node 2 = 3
next for Node 4 = 3
Got value 3
next for Node 3 StopIteration
next for Node 2 StopIteration
next for Node 4 = 4
Got value 4
next for Node 5 = 5
next for Node 6 = 5
next for Node 4 = 5
Got value 5
next for Node 5 StopIteration
next for Node 6 = 6
next for Node 4 = 6
Got value 6
next for Node 7 = 7
next for Node 6 = 7
next for Node 4 = 7
Got value 7
next for Node 7 StopIteration
next for Node 6 StopIteration
next for Node 4 StopIteration

--

You can see that next is called three times (log N) for each value
returned.

It's not about the stack space, it's about going up and down the stack
over and over.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Data Types

[Cai Gengyang]

On Wednesday, September 21, 2016 at 11:22:42 AM UTC+8, Steve D'Aprano wrote:
> On Wed, 21 Sep 2016 01:00 pm, Cai Gengyang wrote:
> 
> > So I am going through chapter 7.1 (Data Types) of
> > http://programarcadegames.com/index.php… What do these terms (input and
> > output) mean --- Can someone explain in plain English and layman's terms ?
> 
> http://www.dictionary.com/browse/input
> 
> http://www.dictionary.com/browse/output
> 
> 
> Does that answer your question about "input and output"?
> 
> 
> > Thanks a lot ...
> > 
>  type(3)
> > 
>  type(3.145)
> > 
>  type("Hi there")
> > 
>  type(True)
> > 
> 
> I don't understand why you are showing this. What part don't you understand?
> 
> The number 3 is an int (short for integer); that is the type of value it is.
> 5 is also an int, and 0, and 9253, and -73. They are all integers.
> 
> 3.145 is a float (short for "floating point number"). "Hi there" is a str
> (short for string). True is a bool (short for Boolean value, which is a
> technical term for special True/False values).
> 
> Perhaps if you can ask a more clear question, I can give a more clear
> answer.
> 
> 
> 
> 
> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.

Right, 

So for example for "bool" , it only applies to True/False and nothing else? (2 
data types), i.e. :

>>> type(True)

>>> type(False)


Are there any other data types that will give you type(A) or type(B) =  besides True and False?

Regards,

GY

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module

[dl l]

Thank you all for the help.

I found the problem is resolved if call PyGC_Collect() after
PyDict_DelItemString(). Is it expected to call PyGC_Collect() here for
Python 3.5 which is not needed for Python 3.3?

2016-09-20 16:01 GMT+08:00 Chris Angelico :

> On Tue, Sep 20, 2016 at 4:19 PM, dl l  wrote:
> > Yes, it's a workaround to set the global variables to None. But My app
> just
> > provide a framework for my customers to run python scripts. That's means
> > the workaround requires my customers to update their python scripts. That
> > may make them unhappy :(. Is there a workaround in my code C++ side to
> call
> > Python C APIs to resolve the memory leak issue?
> >
>
> Python's import system is fundamentally not designed for what I
> believe is your use-case here, of running potentially-edited user
> code. Instead, I suggest keeping 'import' for stable modules that
> don't get updated without restarting the process, and for the more
> dynamic code, build something using exec (or PyRun_StringFlags, which
> I believe is the C equivalent). Object lifetimes will then be entirely
> under your control.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Steve D'Aprano]

On Wed, 21 Sep 2016 12:47 pm, Chris Angelico wrote:

> On Wed, Sep 21, 2016 at 12:34 PM, Steve D'Aprano
>  wrote:
>> "since values from the leaves of the tree have to be yielded
>> multiple times to the top of the tree"
>>
>> Each leaf is yielded once, and I don't understand what you mean by
>> yielding to the top of the tree.
> 
> Ultimately, yes. But AIUI he's talking about this:
> 
> def leaf():
> yield 42
> def intermediate():
> yield next(leaf())
> def root():
> yield next(intermediate())


I don't see that in the OP's code. Nor do I understand where this would be
used. Surely root should be the root of the tree, not a function that walks
a linear chain to yield the bottom of the chain?



> The value 42 gets "passed up the chain" until it gets to the top.

How else do you expect to get to the bottom of the chain starting from the
top except by traversing the chain?

If you don't want to traverse the chain, then don't start at the top.





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Steve D'Aprano]

On Wed, 21 Sep 2016 02:46 am, ROGER GRAYDON CHRISTMAN wrote:

> I am trying to find a better (i.e. more efficient) way to implement a
> generator that traverses a tree.
> 
> The current model of the code (which is also used by a textbook I am
> teaching from does this)
> 
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
> 
> This is nice, simple, and straightforward, but has an O(n log n) running
> time, since
> values from the leaves of the tree have to be yielded multiple times to
> the top of the tree.
> 
> Now, I could certainly implement a linear-time traversal without the
> gnerator:
> 
> def to_list(node,result):
>   """append node information to result"""
>   result = to_list(node._left, result)
>   result.append(node._value)
>   return to_list(node,_right, result)


I forgot to ask... 

How to you reason that this is O(N) time if the same algorithm above is
supposedly O(N log N)? They're both the same algorithm, with the same
performance characteristics. They differ only that the __iter__ version
yields the items one at a time, while the to_list version accumulates them
into a list all at once. But they walk the tree in the same way, touch the
same number of nodes, generate the same depth of calling stack.

You can re-write to_list as:

def to_list(node):
result = []
for value in node:  # calls __iter__ method above
result.append(node)
return result



One of us is badly missing something.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Data Types

[Steve D'Aprano]

On Wed, 21 Sep 2016 01:00 pm, Cai Gengyang wrote:

> So I am going through chapter 7.1 (Data Types) of
> http://programarcadegames.com/index.php… What do these terms (input and
> output) mean --- Can someone explain in plain English and layman's terms ?

http://www.dictionary.com/browse/input

http://www.dictionary.com/browse/output


Does that answer your question about "input and output"?


> Thanks a lot ...
> 
 type(3)
> 
 type(3.145)
> 
 type("Hi there")
> 
 type(True)
> 

I don't understand why you are showing this. What part don't you understand?

The number 3 is an int (short for integer); that is the type of value it is.
5 is also an int, and 0, and 9253, and -73. They are all integers.

3.145 is a float (short for "floating point number"). "Hi there" is a str
(short for string). True is a bool (short for Boolean value, which is a
technical term for special True/False values).

Perhaps if you can ask a more clear question, I can give a more clear
answer.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Steve D'Aprano]

On Wed, 21 Sep 2016 12:44 pm, Random832 wrote:

> On Tue, Sep 20, 2016, at 22:34, Steve D'Aprano wrote:
>> I'm afraid I don't understand this. This is a standard binary tree
>> inorder traversal. Each node is visited once, and there are N nodes,
>> so I make that out to be O(N) not O(N log N). I'm afraid I can't parse
>> your final clause:
>>
>> "since values from the leaves of the tree have to be yielded
>> multiple times to the top of the tree"
>>
>> Each leaf is yielded once, and I don't understand what you mean by
>> yielding to the top of the tree.
> 
> His point is that in order for the top-level iterator to return a given
> node there's a yield call in the top level's iterator , that calls the
> next level's iterator's yield, that calls the next one, so on, in a call
> stack log(n) levels deep.

Right. That's the whole point of a binary search tree. An unbalanced binary
tree may be as deep as N, but a balanced one, or a random unbalanced one,
is only log N deep.

log N is a long way from N log N.

I don't see what is the actual problem we are being asked to solve. Is it
the stack space needed to walk the tree? The time taken? The Original
Poster said:

"O(n log n) running time"

so I don't think the O(log N) stack space used is relevant.


In a practical sense, the right way to solve this problem is to not use a
tree in the first place. But presumably the OP is using this for teaching
about trees, not as production software.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Data Types

[Cai Gengyang]

So I am going through chapter 7.1 (Data Types) of 
http://programarcadegames.com/index.php…
What do these terms (input and output) mean --- Can someone explain in plain 
English and layman's terms ? Thanks a lot ...

>>> type(3)

>>> type(3.145)

>>> type("Hi there")

>>> type(True)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Chris Angelico]

On Wed, Sep 21, 2016 at 12:34 PM, Steve D'Aprano
 wrote:
> "since values from the leaves of the tree have to be yielded
> multiple times to the top of the tree"
>
> Each leaf is yielded once, and I don't understand what you mean by yielding
> to the top of the tree.

Ultimately, yes. But AIUI he's talking about this:

def leaf():
yield 42
def intermediate():
yield next(leaf())
def root():
yield next(intermediate())

The value 42 gets "passed up the chain" until it gets to the top. IMO
the cost of that should be dwarfed by the actual work of whatever's
doing the traversal, but it is potentially a cost. And 'yield from',
being a single opcode, is potentially optimizable (and I believe
CPython does optimize long yield-from chains), so there's no good
reason not to use it - more readable AND potentially faster.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Random832]

On Tue, Sep 20, 2016, at 22:34, Steve D'Aprano wrote:
> I'm afraid I don't understand this. This is a standard binary tree
> inorder traversal. Each node is visited once, and there are N nodes,
> so I make that out to be O(N) not O(N log N). I'm afraid I can't parse
> your final clause:
>
> "since values from the leaves of the tree have to be yielded
> multiple times to the top of the tree"
>
> Each leaf is yielded once, and I don't understand what you mean by
> yielding to the top of the tree.

His point is that in order for the top-level iterator to return a given
node there's a yield call in the top level's iterator , that calls the
next level's iterator's yield, that calls the next one, so on, in a call
stack log(n) levels deep.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Steve D'Aprano]

On Wed, 21 Sep 2016 02:46 am, ROGER GRAYDON CHRISTMAN wrote:

> I am trying to find a better (i.e. more efficient) way to implement a
> generator that traverses a tree.
> 
> The current model of the code (which is also used by a textbook I am
> teaching from does this)
> 
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
> 
> This is nice, simple, and straightforward, but has an O(n log n) running
> time, since
> values from the leaves of the tree have to be yielded multiple times to
> the top of the tree.


I'm afraid I don't understand this. This is a standard binary tree inorder
traversal. Each node is visited once, and there are N nodes, so I make that
out to be O(N) not O(N log N). I'm afraid I can't parse your final clause:

"since values from the leaves of the tree have to be yielded 
multiple times to the top of the tree"

Each leaf is yielded once, and I don't understand what you mean by yielding
to the top of the tree.


I thought that maybe I missed something obvious, so I knocked up a quick and
dirty tree structure, monkey-patched the iter() built-in to count how many
times it was called, and ran your traversal code. Here's my code:


# - cut -


class Node:
def __init__(self, value):
self._left = []  # Sentinel for an empty leaf.
self._right = []
self._value = value
def __iter__(node):
# By the way, you don't have to explicitly call iter() here.
for x in iter(node._left):
yield x
yield node._value
for x in iter(node._right):
yield x


def insert(tree, value):
if value < tree._value:
if tree._left == []:
# add new leaf node
tree._left = Node(value)
else:
insert(tree._left, value)
elif value > tree._value:
if tree._right == []:
tree._right = Node(value)
else:
insert(tree._right, value)


data = list(range(1))
import random
random.shuffle(data)

tree = Node(data[0])
for value in data[1:]:
insert(tree, value)

_iter = iter  # grab a reference to the built-in
count = 0
def iter(obj):  # Monkey-patch the built-in
global count
count += 1
return _iter(obj)


assert list(tree) == sorted(data)
print(count)


# - cut -


which prints 2, as expected: for each node, iter() gets called twice.

So I don't understand where your O(N log N) behaviour is coming from.







-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Lawrence D’Oliveiro]

On Wednesday, September 21, 2016 at 12:14:54 PM UTC+12, Bill Deegan wrote:
> Use Git... at least you get can back what used to work..

That has become such a matter of programming habit, taken so much for granted 
that, to me, it’s like saying “don’t forget to breathe”. :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Idiomatic code validator?

[Tim Johnson]

* Steve D'Aprano  [160920 16:29]:
> On Wed, 21 Sep 2016 01:41 am, Tim Johnson wrote:
> 
> > Not to confuse idiomatic code validation with pep8 validation (I use
> > elpy on emacs)
> > 
> > Is there such a thing as a validator for _idiomatic_ code?
> 
> 
> Yes, it is called a linter. There are various linters available:
> 
> pylint, pychecker, jedi, flake
> 
> 
> there may be others. Depending on how good the linter is, they will do
> things like flag unused arguments, dead code, use of deprecated features,
> and so forth. Some of them may be very opinionated, e.g. flag uses of map()
> as unidiomatic and recommend a list comprehension instead. Whether you
> agree with that opinion or not is up to you.
  Sorry. Should have added this: It might be useful for me to use a more
  "opinionated" linter outside of my "IDE".
  cheers
> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
  :) Again. Sorry!
 
-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Idiomatic code validator?

[Tim Johnson]

* Steve D'Aprano  [160920 16:29]:
> On Wed, 21 Sep 2016 01:41 am, Tim Johnson wrote:
> 
> > Not to confuse idiomatic code validation with pep8 validation (I use
> > elpy on emacs)
> > 
> > Is there such a thing as a validator for _idiomatic_ code?
> 
> 
> Yes, it is called a linter. There are various linters available:
> 
> pylint, pychecker, jedi, flake
  thanks Steve :

  I use flake8 thru elpy. I'm guessing that my inquiry is redundant.
(I might need to look at more fine-grained configuration thru
elpy)

  cheers
  - tj -
> 
> there may be others. Depending on how good the linter is, they will do
> things like flag unused arguments, dead code, use of deprecated features,
> and so forth. Some of them may be very opinionated, e.g. flag uses of map()
> as unidiomatic and recommend a list comprehension instead. Whether you
> agree with that opinion or not is up to you.
> 
> 
> 
> 
> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[ROGER GRAYDON CHRISTMAN]

On Tue, Sep 20, 2016 at 9:46 AM, ROGER GRAYDON CHRISTMAN d...@psu.edu>
 wrote:
>I am trying to find a better (i.e. more efficient) way to implement a generator
>
that traverses a tree.
>
>
The current model of the code (which is also used by a textbook I am teaching
>
from does this)
>
>
   def __iter__(node):
>
 for x in iter(node._left):
>
  yield x
>
 yield node._value
>
 for x in iter(node._right)
>
  yield x
>
>
This is nice, simple, and straightforward, but has an O(n log n) running time,
>
since
>
values from the leaves of the tree have to be yielded multiple times to the top
>
of the tree.
>





Justin Walters  responded:



> Are the left and right attributes collections of more nodes or are they 
> simply references to the node's position in the tree? 
> From the code provided it seems like the former is true and a node's left 
> attribute is a reference to another node?

They are references to other nodes, which in turn have values and left and 
right references to more nodes, etc.
as in a particular tree.  There are no 'collections' in the sense of set or map 
or Python list, just recursive sub-trees.



> I don't know how flexible you are with the structure of your tree, 
but you could try taking the modified pre-order tree traversal approach.



> This article explains it in the context of a database, but the idea is the 
> same:  



> Each node would then have a parent attribute as well as left and 
right attributes. The parent would be a reference to a parent node, and 
the left and right would be integers that position the element in the 
tree. 



> The upside to this is that traversal and lookup is much faster 
since you do not need to have an iterator nested in an iterator. This is
 because the top level node will always have the lowest integer as it's 
left attribute and the highest integer as it's right attribute. That 
means that you can instead have a single iterator that iterates through a
 range of integers to yield the node with the specified left and right 
values.

I am in academia, and the current unit is the binary search tree in its 
conventional sense, instead of the tabular version.
Indeed that table would be faster for iteration, but at the increased cost of 
slowing down adding and removing elements.
If I were to use any array-based model, as this database link shows, all 
insertions and deletions would be linear-time
operations instead of logarithmic time.  

I would contrast the time to insert N elements and then to traverse the tree 
once, where the N elements arrive in any order:
Tabular approach N linear-time insertions -> quadratic time plus a linear-time 
tree traversal afterwards.
Tree approach:   N logarithm-time insertions plus an iteration that takes a 
time proportional to all that -- it still wins.
Since traversal is 'rare', I don't want to penalize the common behaviors to 
accelerate this traversal.

But I would still like to see if there is a linear traversal possible with the 
help if the itertools chain

Roger Christman
instructor
Pennsylvania State University


 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Steve D'Aprano]

On Wed, 21 Sep 2016 07:20 am, Andrew Clark wrote:

> I've restarted my code so many times i no longer have a working version of
> anything.


*Restarting* your code doesn't change it and cannot possibly stop it from
working. My guess is that you actually mean that you've made so many random
edits to your code, without understanding or thinking about what you are
doing, that you've broken it irreversibly.

I think you've learned a few things:

(1) Debugging by making random perturbations to code should be a last
resort, and even then, only done with the greatest of care.

(2) Use version control so you can roll back changes.

(3) Or at least make a backup copy of a known working version of your code.

(4) Most importantly, never make so many changes to your ONLY copy of the
code in one go that you can break it so badly that you can't go back.


You've described your problem quite well, and nicely broken it up into
pieces. I suggest you take each piece, and try to write the code for it
independently of the others. Start with the first piece:

"access remote directories"


Okay, how are they accessible? Are they just network drives? Then that's
simple: you can access them as if they were local directories. What's the
path name of the network drive(s)? Simply use that. Problem solved.

If not, then you need to decide how to access them: over SSH, FTP,
sneaker-net, whatever. The choice you make here is going to impact the
complexity of your code. Think about carefully.

Once you have working code that can list the remote directory, you can use
it to list your three sets of files:


startupfiles = listdir(...StartupConfig)
runningfiles = listdir(...RunningConfig)
archivefiles = listdir(...ArchiveConfig)


now you can move onto step 2:

"run through startup, running and archive to find files 
 with same hostname(do this for all files)"


Now you can forget all about remote file systems (at least for now) and just
work with the three lists of file names. How do you decide which files
belong to which file name? I don't know, because you don't say. Is the
hostname in the file name? Or do you have to open the file and read the
information from the file contents?

However you do it, start by generating a mapping of hostname: list of file
names.


mapping = {}
for filename in list_of_filenames:
hostname = get_hostname(filename)
if hostname in mapping:
mapping[hostname].append(filename)
else:
mapping[hostname] = [filename]  # start a new list with one entry



Once you've done that for all three directories, then you can collect all
the file names for each host from all three directories, and compare the
files.


I leave the rest of the exercise to you, but remember the principle of
Divide And Conquer. Divide the problem into smaller problems which are easy
to solve, solve each small problem, and the big problem solves itself.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list or dictionary

[Steve D'Aprano]

On Wed, 21 Sep 2016 04:04 am, Ganesh Pal wrote:

> I am on python 2.7 and Linux
> 
> I have the stdout in the below form , I need to write a function to get
> hostname for the given id.
> 
> 
> Example:
> 
 stdout
> 'hostname-1 is array with id 1\nhostname-2 is array with  id 2\nhostname-3
> is array with id 3\n'
> 
> 
> def get_hostname(id)
>return id
> 
> what's a better option here
> 
> 1. store it in list and grep for id and return
> 2. store it in dict as key and value i.e hostname = { 'hostname1': 1} and
> return key
> 3. any other simple options.


Why don't you write a function for each one, and see which is less work?

# Version 1: store the hostname information in a list as strings

hostnames = ['hostname-1 is array with id 1', 
 'hostname-2 is array with id 2',
 'hostname-842 is array with id 842']

def get_hostname(id):
for line in hostnames:
regex = r'(.*) is array with id %d' % id
mo = re.match(regex, line)
if mo is not None:
return mo.group(1)
raise ValueError('not found')


# Version 2: store the hostname information in a dict {hostname: id}

hostnames = {'hostname1': 1, 'hostname2': 2, 'hostname842': 842}

def get_hostname(id):
for key, value in hostnames.items():
if value == id:
return key
raise ValueError('not found')


# Version 3: use a dict the way dicts are meant to be used

hostnames = {1: 'hostname1', 2: 'hostname2', 842: 'hostname842'}

def get_hostname(id):
return hostnames[id]








-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Idiomatic code validator?

[Steve D'Aprano]

On Wed, 21 Sep 2016 01:41 am, Tim Johnson wrote:

> Not to confuse idiomatic code validation with pep8 validation (I use
> elpy on emacs)
> 
> Is there such a thing as a validator for _idiomatic_ code?


Yes, it is called a linter. There are various linters available:

pylint, pychecker, jedi, flake


there may be others. Depending on how good the linter is, they will do
things like flag unused arguments, dead code, use of deprecated features,
and so forth. Some of them may be very opinionated, e.g. flag uses of map()
as unidiomatic and recommend a list comprehension instead. Whether you
agree with that opinion or not is up to you.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Bill Deegan]

Use Git... at least you get can back what used to work..

On Tue, Sep 20, 2016 at 2:20 PM, Andrew Clark  wrote:

> On Tuesday, September 20, 2016 at 3:25:11 PM UTC-5, Lawrence D’Oliveiro
> wrote:
> > On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark
> wrote:
> > > If anyone can help me out with sudo code or set me in the right
> direction
> > > would be nice.
> >
> > You know What Aesop said: “the gods* help those who help themselves”.
> Start by posting an initial stab at your code for solving, if not the whole
> problem, at least part of it.
> >
> > *for “gods”, read “peanut gallery”. They’ll be falling over themselves
> to critique every little shortcoming, real or otherwise, in the code you
> post.
>
> Thanks. I've restarted my code so many times i no longer have a working
> version of anything. I'm trying to get the "go to remote directory" part to
> get files done first.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to automate java application in window using python

[Matt Wheeler]

On Tue, 20 Sep 2016, 02:47 meInvent bbird,  wrote:

> can it contorl Maplesoft's maple which is a java executable file?
>

I don't know maple so I can't answer that.

Which programming language an application is written in isn't really
relevant for pywinauto, it's the graphical toolkit in use which will make a
difference.
>From the one screenshot I found during a quick look at the Maplesoft site,
it looks like maple is using a toolkit which presents native Win32 GUI API
controls, so my *guess* is that it will work, but you'll have to try it to
find out.

I would suggest installing pywinauto and importing it in an interactive
python shell...

>>> import pywinauto
>>> maple = pywinauto.application.Application().start('maple')
>>> maple.Maple.PrintControlIdentifiers()

This will output every control on the window named "Maple" which belongs to
the 'maple' executable which was launched in the step before.
If it outputs lots of stuff you're in luck, you now get to dig through that
output to find the controls you're interested in :)

>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the sum of differences between integers in a list

[John Pote]

On 20/09/2016 12:52, Daiyue Weng wrote:


Hi, I have a list numbers say,

[1,2,3,4,6,8,9,10,11]

First, I want to calculate the sum of the differences between the numbers
in the list.
At least for this first part a little pencil, paper and algebra yields a 
simple formula of constant and minimal calculation time. I had an 
intuitive guess and wrote down the sum of differences for a couple of 
examples,

[1, 2, 5]   => 4
[9, 11, 17, 19] => 10
It works for negative differences as well,
[1, 2, 5, 1]=> 0
The trick is to spot the relation between the sum of differences and the 
numbers in the list. A few lines of algebra then gave a very simple formula.


As for the rest it's down to code as others have hinted at.

Second, if a sequence of numbers having a difference of 1, put them in a
list, i.e. there are two such lists,

[1,2,3]

[8,9,10,11]

and also put the rest numbers in another list, i.e. there is only one such
list in the example,

[6].

Third, get the lists with the max/min sizes from above, i.e. in this
example, the max list is,

[8,9,10,11]

min list is

[1,2,3].

What's the best way to implement this?

cheers


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to install Python.h on FreeBSD 10.3-RELEASE?

[The Doctor]

In article <201609...@crcomp.net>, Don Kuenz   wrote:
>
>The Doctor  wrote:
>> In article <201609...@crcomp.net>, Don Kuenz   wrote:
>>>
>>>It turns out that the question isn't "How to install Python.h?" The
>>>question is "Why doesn't make find Python.h?"
>>>
>>>
>>>
>>># uname -v
>>>FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014
>>>
>>># cd /
>>># find . -name 'Python.h'
>>>/usr/local/include/python2.7/Python.h
>>>/usr/local/include/python3.4m/Python.h
>>>
>>>
>
>
>
>>>
>>>It seems that a symbolic link may be needed. Fortunately the developer
>>>is working with me on this problem. He may know the best way to proceed.
>>
>> Which version of Python do you wish to use?
>
>It's best for me to know how to use either version. It's OK if each
>version is mutually exclusive.
>
>Thank you,
>
>--
>Don Kuenz KB7RPU
>
>Every tool carries with it the spirit by which it has been created.
> - Heisenberg

python --version.
-- 
Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! 
http://www.fullyfollow.me/rootnl2k  Look at Psalms 14 and 53 on Atheism
Time for the USA to hold a referendum on its republic and vote to dissolve!! 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pypy on windows much slower than linux/mac when using complex number type?

[Chris Kaynor]

On Tue, Sep 20, 2016 at 3:59 PM, Chris Angelico  wrote:

> On Wed, Sep 21, 2016 at 8:50 AM, Irmen de Jong 
> wrote:
> >> Dunno if it's the cause or not, but you're running a 32-bit PyPy on a
> >> 64-bit Windows. I could well imagine that that has some odd
> >> significance.
> >>
> >> ChrisA
> >
> >
> > Perhaps. Though I can't really imagine what's going on there then. The
> one on Linux is
> > 32 bits as well and it's also much faster...
> > Unfortunately there's no 64 bits pypy expected for windows, so I can't
> test that
>
> Yeah, but the Linux one is running in a 32-bit OS. I don't know how
> the "32-bit on 64-bit" subsystem of Windows works and how fast it is;
> it could be that the thunking defeats some optimizations relating to
> floating-point. Who knows. Hard to test.
>

WoW64 shouldn't have any measurable speed decrease. I've run plenty of
32-bit games on a 64-bit Windows (many games are still 32-bit only), and
they always perform fine, regardless of whether they are CPU or GPU bound
based off the profiling I've done. I think most of the cost is paid in the
context switches.

Regarding the performance decrease, it may be worthwhile to push the report
to a PyPy specific forum - a PyPy developer will probably see it here, but
you may get a faster response on a forum specific to PyPy.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pypy on windows much slower than linux/mac when using complex number type?

[Chris Angelico]

On Wed, Sep 21, 2016 at 8:50 AM, Irmen de Jong  wrote:
>> Dunno if it's the cause or not, but you're running a 32-bit PyPy on a
>> 64-bit Windows. I could well imagine that that has some odd
>> significance.
>>
>> ChrisA
>
>
> Perhaps. Though I can't really imagine what's going on there then. The one on 
> Linux is
> 32 bits as well and it's also much faster...
> Unfortunately there's no 64 bits pypy expected for windows, so I can't test 
> that

Yeah, but the Linux one is running in a 32-bit OS. I don't know how
the "32-bit on 64-bit" subsystem of Windows works and how fast it is;
it could be that the thunking defeats some optimizations relating to
floating-point. Who knows. Hard to test.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pypy on windows much slower than linux/mac when using complex number type?

[Irmen de Jong]

On 20-9-2016 22:43, Chris Angelico wrote:
> On Wed, Sep 21, 2016 at 6:38 AM, Irmen de Jong  wrote:
>>  Windows: 64 bits Windows 7, Intel Core 2 Quad 3.4 Ghz
>>  Linux: 32 bits Mint 18, Virtualbox VM on above windows machine
>>  Mac mini: OS X 10.11.6, Intel Core 2 Duo 2.53 Ghz
>>
>> The test code I've been using is here:
>>  https://gist.github.com/irmen/c6b12b4cf88a6a4fcf5ff721c7089078
>>
>> Test results:
>>   function:  mandel   / iterations
>>  Mac mini, Pypy 5.4.1 (64-bit):  0.81 sec / 0.65 sec
>>  Linux, Pypy 5.1 (32-bit):   1.06 sec / 0.64 sec
>>  Windows, Pypy 5.4.1 (32-bit):   5.59 sec / 2.87 sec
>>
>>
>> What could cause such a huge difference?
> 
> Dunno if it's the cause or not, but you're running a 32-bit PyPy on a
> 64-bit Windows. I could well imagine that that has some odd
> significance.
> 
> ChrisA


Perhaps. Though I can't really imagine what's going on there then. The one on 
Linux is
32 bits as well and it's also much faster...
Unfortunately there's no 64 bits pypy expected for windows, so I can't test that

Irmen


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to install Python.h on FreeBSD 10.3-RELEASE?

[Don Kuenz]

The Doctor  wrote:
> In article <201609...@crcomp.net>, Don Kuenz   wrote:
>>
>>It turns out that the question isn't "How to install Python.h?" The
>>question is "Why doesn't make find Python.h?"
>>
>>
>>
>># uname -v
>>FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014
>>
>># cd /
>># find . -name 'Python.h'
>>/usr/local/include/python2.7/Python.h
>>/usr/local/include/python3.4m/Python.h
>>
>>



>>
>>It seems that a symbolic link may be needed. Fortunately the developer
>>is working with me on this problem. He may know the best way to proceed.
>
> Which version of Python do you wish to use?

It's best for me to know how to use either version. It's OK if each
version is mutually exclusive.

Thank you,

--
Don Kuenz KB7RPU

Every tool carries with it the spirit by which it has been created.
 - Heisenberg
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Andrew Clark]

On Tuesday, September 20, 2016 at 3:25:11 PM UTC-5, Lawrence D’Oliveiro wrote:
> On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark wrote:
> > If anyone can help me out with sudo code or set me in the right direction
> > would be nice.
> 
> You know What Aesop said: “the gods* help those who help themselves”. Start 
> by posting an initial stab at your code for solving, if not the whole 
> problem, at least part of it.
> 
> *for “gods”, read “peanut gallery”. They’ll be falling over themselves to 
> critique every little shortcoming, real or otherwise, in the code you post.

Thanks. I've restarted my code so many times i no longer have a working version 
of anything. I'm trying to get the "go to remote directory" part to get files 
done first.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list or dictionary

[Chris Angelico]

On Wed, Sep 21, 2016 at 4:04 AM, Ganesh Pal  wrote:
> 1. store it in list and grep for id and return
> 2. store it in dict as key and value i.e hostname = { 'hostname1': 1} and
> return key
> 3. any other simple options.

4. Store it in dict, but the other way around. The key should be the
thing you want to look up; the value, the thing you want to find out.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Chris Angelico]

On Wed, Sep 21, 2016 at 6:21 AM, Martin Schöön  wrote:
> Den 2016-09-19 skrev Lawrence D’Oliveiro :
>> On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
>>> But -- now I tested using emacs instead using C-c C-c to execute.
>>> Noting happens so I try to run the program from command line and
>>> find that now Python can't stand my å, ä and ö.
>>
>> What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course.
>
> Python 2.7.
> I just tried running my code in Python 3 and that worked like charm.

Then you've found the solution. Py2's Unicode support is weaker than
Py3's, and it often depends on encodings.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pypy on windows much slower than linux/mac when using complex number type?

[Chris Angelico]

On Wed, Sep 21, 2016 at 6:38 AM, Irmen de Jong  wrote:
>  Windows: 64 bits Windows 7, Intel Core 2 Quad 3.4 Ghz
>  Linux: 32 bits Mint 18, Virtualbox VM on above windows machine
>  Mac mini: OS X 10.11.6, Intel Core 2 Duo 2.53 Ghz
>
> The test code I've been using is here:
>  https://gist.github.com/irmen/c6b12b4cf88a6a4fcf5ff721c7089078
>
> Test results:
>   function:  mandel   / iterations
>  Mac mini, Pypy 5.4.1 (64-bit):  0.81 sec / 0.65 sec
>  Linux, Pypy 5.1 (32-bit):   1.06 sec / 0.64 sec
>  Windows, Pypy 5.4.1 (32-bit):   5.59 sec / 2.87 sec
>
>
> What could cause such a huge difference?

Dunno if it's the cause or not, but you're running a 32-bit PyPy on a
64-bit Windows. I could well imagine that that has some odd
significance.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

pypy on windows much slower than linux/mac when using complex number type?

[Irmen de Jong]

Hi,

I've stumbled across a peculiar performance issue with Pypy across some 
different
platforms. It was very visible in some calculation heavy code that I wrote that 
uses
Python's complex number type to calculate the well-known Mandelbrot set.

Pypy running the code on my Windows machine is doing okay, but when running the 
same
code on Pypy on different systems, the performance difference is so big it is 
not even
funny. The other implementations are MUCH faster than the windows one. Which is 
quite
unexpected because the other machines I've tested on have the same or much lower
physical CPU specs than the windows machine.  Here's the comparison:

Machine specs:
 Windows: 64 bits Windows 7, Intel Core 2 Quad 3.4 Ghz
 Linux: 32 bits Mint 18, Virtualbox VM on above windows machine
 Mac mini: OS X 10.11.6, Intel Core 2 Duo 2.53 Ghz

The test code I've been using is here:
 https://gist.github.com/irmen/c6b12b4cf88a6a4fcf5ff721c7089078

Test results:
  function:  mandel   / iterations
 Mac mini, Pypy 5.4.1 (64-bit):  0.81 sec / 0.65 sec
 Linux, Pypy 5.1 (32-bit):   1.06 sec / 0.64 sec
 Windows, Pypy 5.4.1 (32-bit):   5.59 sec / 2.87 sec


What could cause such a huge difference?

Is it perhaps a compiler issue (where gcc/clang are MUCH better at optimizing 
certain
things, although I wonder how much of a factor this is because Pypy is doing 
JITting by
itself as far as I am aware)?   Or is something strange going on with the way 
the
complex number type is implemented?   (the difference doesn't occur when using 
only floats)


Regards
Irmen de Jong

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Peter Otten]

Martin Schöön wrote:

> Den 2016-09-19 skrev Christian Gollwitzer :
>> Am 19.09.16 um 22:21 schrieb Martin Schöön:
>>> I am studying some of these tutorials:
>>> https://pythonprogramming.net/matplotlib-intro-tutorial/
>>>
>>> I am recreating the code and I use my native Swedish for comments,
>>> labels and titles. Until now I have been doing so using Geany
>>> and executing using F5. This works fine.
>>>
>>> But -- now I tested using emacs instead using C-c C-c to execute.
>>> Noting happens so I try to run the program from command line and
>>> find that now Python can't stand my å, ä and ö.
>>>
>>> I am puzzled: With Geany there is no problem but outside Geany
>>> I am punished by Python for using Swedish.
>>
>> you are not "punished for Swedish", you need to tell Python the encoding
>> of the file it runs. See here:
>> https://www.python.org/dev/peps/pep-0263/
>>
>> Assuming that you use UTF-8 (you should check with an emacs expert, I am
>> not an emacs user), try putting the header
>>
>> #!/usr/bin/python
>> # -*- coding: utf-8 -*-
>>
>> on top of your files.
>>
> I already have this and since it doesn't work from command line
> either it can't be an emacs unique problem.
> 
> Still confused...
> 
> Too late to find a emacs forum tonight.
> 
> /Martin

Are all non-ascii strings unicode? I. e.

u"Schöön" rather than just "Schöön"

?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: list or dictionary

[Lawrence D’Oliveiro]

On Wednesday, September 21, 2016 at 6:05:41 AM UTC+12, Ganesh Pal wrote:
> I am on python 2.7 ...

Why?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Lawrence D’Oliveiro]

On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark wrote:
> If anyone can help me out with sudo code or set me in the right direction
> would be nice.

You know What Aesop said: “the gods* help those who help themselves”. Start by 
posting an initial stab at your code for solving, if not the whole problem, at 
least part of it.

*for “gods”, read “peanut gallery”. They’ll be falling over themselves to 
critique every little shortcoming, real or otherwise, in the code you post.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-19 skrev Lawrence D’Oliveiro :
> On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote:
>> But -- now I tested using emacs instead using C-c C-c to execute.
>> Noting happens so I try to run the program from command line and
>> find that now Python can't stand my å, ä and ö.
>
> What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course.

Python 2.7.
I just tried running my code in Python 3 and that worked like charm.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Martin Schöön]

Den 2016-09-19 skrev Christian Gollwitzer :
> Am 19.09.16 um 22:21 schrieb Martin Schöön:
>> I am studying some of these tutorials:
>> https://pythonprogramming.net/matplotlib-intro-tutorial/
>>
>> I am recreating the code and I use my native Swedish for comments,
>> labels and titles. Until now I have been doing so using Geany
>> and executing using F5. This works fine.
>>
>> But -- now I tested using emacs instead using C-c C-c to execute.
>> Noting happens so I try to run the program from command line and
>> find that now Python can't stand my å, ä and ö.
>>
>> I am puzzled: With Geany there is no problem but outside Geany
>> I am punished by Python for using Swedish.
>
> you are not "punished for Swedish", you need to tell Python the encoding 
> of the file it runs. See here:
> https://www.python.org/dev/peps/pep-0263/
>
> Assuming that you use UTF-8 (you should check with an emacs expert, I am 
> not an emacs user), try putting the header
>
> #!/usr/bin/python
> # -*- coding: utf-8 -*-
>
> on top of your files.
>
I already have this and since it doesn't work from command line
either it can't be an emacs unique problem.

Still confused...

Too late to find a emacs forum tonight.

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[Peter Otten]

Nobody wrote:

> On Tue, 20 Sep 2016 15:12:39 +0200, Peter Otten wrote:
> 
>> because they don't build lists only to throw them away.
> 
> The lists could have been avoided by using iterators, e.g.
> 
> import itertools as it
> keys = xrange(256)
> vals = it.imap(chr, keys)
> max(it.imap(operator.setitem, it.repeat(d), keys, vals))
> 
>> Also, creating a list of dicts or lists is a common gotcha because after
>> 
>> outer = [[]] * 3
>> 
>> the outer list contains *the* *same* list three times, not three empty
>> lists.
> 
> But in this case, it's not a gotcha; 

Remember that the OP is a newbie

> it's intentional 

and the author of the original example is trying to be clever, but in my 
eyes doesn't succeed.

> that each iteration
> operates upon the same dictionary.
> 
> Essentially, [d]*len(keys) is a trick to get around the fact that map()
> requires all of the arguments (apart from the function) to be sequences.
> 
> itertools.repeat() is possibly a better trick, although then you can't use
> map(), because map() iterates until *all* sequences are exhausted,
> appending None values for shorter sequences. itertools.imap() terminates
> once the shortest sequence is exhausted.
> 
> In this specific case, a loop or comprehension would have been better. But
> in situations where you don't control the iteration, the ability to coerce
> something into a pre-determined iteration pattern is useful.

You mean there is a situation where you'd actually recommend/use

> max(it.imap(operator.setitem, it.repeat(d), keys, vals))

? I'd like to see that use case.

On second thought -- we might both settle on "Recommended by nobody" ;)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to automate java application in window using python

[Lawrence D’Oliveiro]

On Sunday, September 18, 2016 at 10:42:16 PM UTC+12, Paul Rubin wrote:
>
> Lawrence D’Oliveiro writes:
>>
> I'm quite sure there are Java bindings for all those protocols.

Are any of these supported by the Java app in question?

Doesn’t seem like it.

>> Like I said, trying to automate a GUI is a waste of time. GUIs are
>> designed for humans, not computers, to use.
> 
> Automation doesn't simulate button presses or anything like that: the
> automate objects expose higher level user actions.  E.g. the web browser
> object has a navigate method and that sort of thing.

In other words, the GUI becomes less and less relevant to this use case.

> The classic automation example is embedding a chunk of an Excel
> spreadsheet in the middle of a Word document, so it's displayed with
> Word's fonts and formatting, but when you change a number in the
> spreadsheet segment, the formulas run and the other numbers change.
> What happens there is Word collects the numbers you type, then calls the
> Excel automation interfaces to update the relevant spreadsheet cells and
> read back new numbers.  There are various hacks in KDE, Gnome, etc.  to
> do similar things under Linux.  It's all transparent to the user and
> presents useful features.

That’s not “automation”, that’s “compound documents”. For some reason this 
facility gets very little use these days.

>> GUIs are the end of the abstraction chain: you cannot build anything
>> more on top of them.
> 
> IMHO you're not contributing useful insights through these incorrect
> guesses about how Windows automation works.

Has anybody contributed a non-problematic solution yet?

Precisely.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to automate java application in window using python

[Lawrence D’Oliveiro]

On Tuesday, September 20, 2016 at 1:11:20 PM UTC+12, Ned Batchelder wrote:
> We get it, you don't like GUIs.

Who says I don’t like GUIs 
?

I just assume we’ve moved on from the 1990s, when they were considered to be 
the pinnacle of computing evolution, that’s all.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Idiomatic code validator?

[Tim Johnson]

* Terry Reedy  [160920 11:48]:
> On 9/20/2016 11:41 AM, Tim Johnson wrote:
> > Not to confuse idiomatic code validation with pep8 validation
> 
> Strictly speaking, there cannot be a mechanical PEP 8 validator, as any
> mechanical checker violates the admonitions of the beginning sections
> 'Introductions' and 'A Foolish Consistency is the Hobgoblin of Little
> Minds'.  For this reason, the core developers request the author of the
> checker formerly known as 'pep8' to change its name, which he did.
  elpy uses flake8
  https://github.com/jorgenschaefer/elpy/wiki/Configuration

> > Is there such a thing as a validator for _idiomatic_ code?
> 
> What would such a thing do?  Flag supposedly non-idiomatic code?  Or mark
> code recognized as 'idiomatic', with no implication either way about other
> code?
  Why not? 

> > I have Knupp's "Writing Idiomatic Python" and have bookmarked some
> > advisory websites that illustrate idiomatic style.
> 
> Do they all agree on 'idiomatic'?
  Not likely, I would not expect them to.
  Thanks

-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

automated comparison tool

[Andrew Clark]

I'm trying to develop a tool in python that will allow me to compare two file. 
I know this sounds like it's been done before but there is a catch.

Files are stored on a remote server.
There are three separate directories.
StartupConfig, RunningConfig, and ArchiveConfig

I need to have the python program do the following

access remote directories

run through startup, running and archive to find files with same hostname(do 
this for all files)

once all three are found compare them to each other returning true or false if 
they are or are not the same

however, files contain certain blocks of text that need to be ignored as these 
blocks of text will never be the same.

I've looked at a couple different libraries such as the difflib and even 
ciscoconfparse.

I'm not new to programming just have not done it for 10+ years. If anyone can 
help me out with sudo code or set me in the right direction would be nice.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Idiomatic code validator?

[Terry Reedy]

On 9/20/2016 11:41 AM, Tim Johnson wrote:

Not to confuse idiomatic code validation with pep8 validation


Strictly speaking, there cannot be a mechanical PEP 8 validator, as any 
mechanical checker violates the admonitions of the beginning sections 
'Introductions' and 'A Foolish Consistency is the Hobgoblin of Little 
Minds'.  For this reason, the core developers request the author of the 
checker formerly known as 'pep8' to change its name, which he did.



Is there such a thing as a validator for _idiomatic_ code?


What would such a thing do?  Flag supposedly non-idiomatic code?  Or 
mark code recognized as 'idiomatic', with no implication either way 
about other code?



I have Knupp's "Writing Idiomatic Python" and have bookmarked some
advisory websites that illustrate idiomatic style.


Do they all agree on 'idiomatic'?  I am sure that the long-time frequent 
contributors on python list do not.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[Nobody]

On Tue, 20 Sep 2016 15:12:39 +0200, Peter Otten wrote:

> because they don't build lists only to throw them away.

The lists could have been avoided by using iterators, e.g.

import itertools as it
keys = xrange(256)
vals = it.imap(chr, keys)
max(it.imap(operator.setitem, it.repeat(d), keys, vals))

> Also, creating a list of dicts or lists is a common gotcha because after
> 
> outer = [[]] * 3
> 
> the outer list contains *the* *same* list three times, not three empty
> lists.

But in this case, it's not a gotcha; it's intentional that each iteration
operates upon the same dictionary.

Essentially, [d]*len(keys) is a trick to get around the fact that map()
requires all of the arguments (apart from the function) to be sequences.

itertools.repeat() is possibly a better trick, although then you can't use
map(), because map() iterates until *all* sequences are exhausted,
appending None values for shorter sequences. itertools.imap() terminates
once the shortest sequence is exhausted.

In this specific case, a loop or comprehension would have been better. But
in situations where you don't control the iteration, the ability to coerce
something into a pre-determined iteration pattern is useful.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[Terry Reedy]

On 9/20/2016 9:12 AM, Peter Otten wrote:


在 2016年9月20日星期二 UTC-4上午8:17:13，BartC写道：

On 20/09/2016 13:12, 38016226...@gmail.com wrote:

d = {}
keys = range(256)
vals = map(chr, keys)
map(operator.setitem, [d]*len(keys), keys, vals)


It is from python library. What does [d]*len(keys) mean?
d is the name of dict but put d in [] really confused me.


Where in which 'python library?  I cannot findI the above in 2.7 or 3.6 
stdlib.  The code should be replaced by



It should be noted that the code above is really bad Python.
Better alternatives are the simple loop

d = {}
for i in range(256):
 d[i] = chr(i)

or the dict comprehension

d = {i: chr(i) for i in range(256)}


this.

--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

list or dictionary

[Ganesh Pal]

I am on python 2.7 and Linux

I have the stdout in the below form , I need to write a function to get
hostname for the given id.


Example:

>>> stdout
'hostname-1 is array with id 1\nhostname-2 is array with  id 2\nhostname-3
is array with id 3\n'


def get_hostname(id)
   return id

what's a better option here

1. store it in list and grep for id and return
2. store it in dict as key and value i.e hostname = { 'hostname1': 1} and
return key
3. any other simple options.



Regards,
Ganesh
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Peter Otten]

ROGER GRAYDON CHRISTMAN wrote:

> I am trying to find a better (i.e. more efficient) way to implement a
> generator that traverses a tree.
> 
> The current model of the code (which is also used by a textbook I am
> teaching from does this)
> 
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
> 
> This is nice, simple, and straightforward, but has an O(n log n) running
> time, since
> values from the leaves of the tree have to be yielded multiple times to
> the top of the tree.
> 
> Now, I could certainly implement a linear-time traversal without the
> gnerator:
> 
> def to_list(node,result):
>   """append node information to result"""
>   result = to_list(node._left, result)
>   result.append(node._value)
>   return to_list(node,_right, result)
> 
> but then that requires the extra memory space to build the list into,
> which is basically what the generator is supposed to avoid.
> 
> Now, I did see that itertools has a chain method for concatenating
> iterators, so it would be nice to concatenate the results from the
> recursive calls without the for loops, but I have no idea how to
> squeeze the 'yield node._value' in between them.

Your above method would become

def __iter__(self):
return chain(self._left, [self._value], self._right)

i. e. you wrap the value in an iterable. 

But I don't see how this could help in terms of big O.
 
> Is there hope for a linear-time tree-traversal generator, or will
> I have just have to settle for an n-log-n generator or a linear time
> behavior with linear extra space?
> 
> Roger Christman
> instructor
> Pennsylvania State University


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[justin walters]

On Tue, Sep 20, 2016 at 9:46 AM, ROGER GRAYDON CHRISTMAN 
wrote:

> I am trying to find a better (i.e. more efficient) way to implement a
> generator
> that traverses a tree.
>
> The current model of the code (which is also used by a textbook I am
> teaching
> from does this)
>
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
>
> This is nice, simple, and straightforward, but has an O(n log n) running
> time,
> since
> values from the leaves of the tree have to be yielded multiple times to
> the top
> of the tree.
>


Are the left and right attributes collections of more nodes or are they
simply references to the node's position in the tree?

>From the code provided it seems like the former is true and a node's left
attribute is a reference to another node?

I don't know how flexible you are with the structure of your tree, but you
could try taking the modified pre-order tree traversal approach.

This article explains it in the context of a database, but the idea is the
same: https://www.sitepoint.com/hierarchical-data-database-2/

Each node would then have a parent attribute as well as left and right
attributes. The parent would be a reference to a parent node, and the left
and right would be integers that position the element in the tree.

The upside to this is that traversal and lookup is much faster since you do
not need to have an iterator nested in an iterator. This is because the top
level node will always have the lowest integer as it's left attribute and
the highest integer as it's right attribute. That means that you can
instead have a single iterator that iterates through a range of integers to
yield the node with the specified left and right values.

The downside is that inserting a new node can take a long time because,
depending on the insertion point, the left and right values for each node
in the tree may have to be recalculated.

Now, I may have completely missed the mark here. I am completely self
taught and have only been programming for about 3 years. I hope that you
gleaned some value from my response either way.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the sum of differences between integers in a list

[mm0fmf]

On 20/09/2016 12:52, Daiyue Weng wrote:


What's the best way to implement this?


With a python script.

Show your work and people will help you.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Rob Gaddi]

ROGER GRAYDON CHRISTMAN wrote:

> I am trying to find a better (i.e. more efficient) way to implement a 
> generator
> that traverses a tree.
>
> The current model of the code (which is also used by a textbook I am teaching
> from does this)
>
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
>
> This is nice, simple, and straightforward, but has an O(n log n) running time,
> since
> values from the leaves of the tree have to be yielded multiple times to the 
> top
> of the tree.
>
> Now, I could certainly implement a linear-time traversal without the gnerator:
>
> def to_list(node,result):
>   """append node information to result"""
>   result = to_list(node._left, result)
>   result.append(node._value)
>   return to_list(node,_right, result)
>
> but then that requires the extra memory space to build the list into, which
> is basically what the generator is supposed to avoid.
>
> Now, I did see that itertools has a chain method for concatenating
> iterators, so it would be nice to concatenate the results from the 
> recursive calls without the for loops, but I have no idea how to
> squeeze the 'yield node._value' in between them.
>
> Is there hope for a linear-time tree-traversal generator, or will
> I have just have to settle for an n-log-n generator or a linear time
> behavior with linear extra space?
>
> Roger Christman
> instructor
> Pennsylvania State University
>

The only thing that's O(N log N) in that is the number of actual yield
calls.  If you're doing pretty much anything with those values as
they're being iterated over then they'll dominate the timing, and that
is O(N).

If you're using Python 3.4 or above you can turn that into

def __iter__(node):
  yield from iter(node._left)
  yield node._value
  yield from iter(node._right)

I think there's a little bit of optimization that goes on using yield
from.  That said, all this has a serious stink of premature
optimization.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linear Time Tree Traversal Generator

[Chris Angelico]

On Wed, Sep 21, 2016 at 2:46 AM, ROGER GRAYDON CHRISTMAN  wrote:
> The current model of the code (which is also used by a textbook I am teaching
> from does this)
>
>def __iter__(node):
>  for x in iter(node._left):
>   yield x
>  yield node._value
>  for x in iter(node._right)
>   yield x
>
> This is nice, simple, and straightforward, but has an O(n log n) running time,
> since
> values from the leaves of the tree have to be yielded multiple times to the 
> top
> of the tree.
>

Normally I'd implement it thus:

def __iter__(self):
if self._left: yield from self._left
yield self._value
if self._right: yield from self._right

But as far as I can see, it's equivalent to your code, except that I
allow the left and right nodes to be None (you might have something
iterable-but-empty in leaf nodes - not sure).

The 'yield from' operation is highly optimized, so it's not going to
cost too much. You don't have to concern yourself overly with the
"yielding up the chain" cost here, as it'll all be done for you by the
interpreter. It's entirely possible for the interpreter to actually
eliminate the chain by storing a reference to the deepest generator;
in any case, it's adequately fast for everything I've ever needed of
it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Linear Time Tree Traversal Generator

[ROGER GRAYDON CHRISTMAN]

I am trying to find a better (i.e. more efficient) way to implement a generator
that traverses a tree.

The current model of the code (which is also used by a textbook I am teaching
from does this)

   def __iter__(node):
 for x in iter(node._left):
  yield x
 yield node._value
 for x in iter(node._right)
  yield x

This is nice, simple, and straightforward, but has an O(n log n) running time,
since
values from the leaves of the tree have to be yielded multiple times to the top
of the tree.

Now, I could certainly implement a linear-time traversal without the gnerator:

def to_list(node,result):
  """append node information to result"""
  result = to_list(node._left, result)
  result.append(node._value)
  return to_list(node,_right, result)

but then that requires the extra memory space to build the list into, which
is basically what the generator is supposed to avoid.

Now, I did see that itertools has a chain method for concatenating
iterators, so it would be nice to concatenate the results from the 
recursive calls without the for loops, but I have no idea how to
squeeze the 'yield node._value' in between them.

Is there hope for a linear-time tree-traversal generator, or will
I have just have to settle for an n-log-n generator or a linear time
behavior with linear extra space?

Roger Christman
instructor
Pennsylvania State University


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the sum of differences between integers in a list

[Peter Otten]

Daiyue Weng wrote:

> Hi, I have a list numbers say,
> 
> [1,2,3,4,6,8,9,10,11]
> 
> First, I want to calculate the sum of the differences between the numbers
> in the list.
> Second, if a sequence of numbers having a difference of 1, put them in a
> list, i.e. there are two such lists,
> 
> [1,2,3]

Why isn't 4 in that list?

> [8,9,10,11]
> 
> and also put the rest numbers in another list, i.e. there is only one such
> list in the example,
> 
> [6].
> 
> Third, get the lists with the max/min sizes from above, i.e. in this
> example, the max list is,
> 
> [8,9,10,11]
> 
> min list is
> 
> [1,2,3].

In addition to Steve's hint: max() takes a key argument, e. g. to get the 
string with the highest amount of "a"s:

>>> max(["xyz", "abc", "alpha"], key=lambda s: s.count("a"))
'alpha'

Finding the longest string or list doesn't even require you to define your 
own function.

-- 
https://mail.python.org/mailman/listinfo/python-list

Idiomatic code validator?

[Tim Johnson]

Not to confuse idiomatic code validation with pep8 validation (I use
elpy on emacs)

Is there such a thing as a validator for _idiomatic_ code?

I have Knupp's "Writing Idiomatic Python" and have bookmarked some
advisory websites that illustrate idiomatic style.

thanks
-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to install Python.h on FreeBSD 10.3-RELEASE?

[The Doctor]

In article <201609...@crcomp.net>, Don Kuenz   wrote:
>
>It turns out that the question isn't "How to install Python.h?" The
>question is "Why doesn't make find Python.h?"
>
>
>
># uname -v
>FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014
>
># cd /
># find . -name 'Python.h'
>/usr/local/include/python2.7/Python.h
>/usr/local/include/python3.4m/Python.h
>
>
>
>$ uname -v
>FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016
>
># cd /
># find . -name 'Python.h'
>/usr/local/include/python2.7/Python.h
>/usr/local/include/python3.4m/Python.h
>
>
>
>Making all in src/cpython
>make  all-am
>depbase=`echo data_arc.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`; /bin/sh
>In file included from data_arc.c:19:0:
>data.h:22:20: fatal error: Python.h: No such file or directory
> #include 
>^
>compilation terminated.
>*** Error code 1
>
>
>
>It seems that a symbolic link may be needed. Fortunately the developer
>is working with me on this problem. He may know the best way to proceed.
>
>Thank you,
>
>--
>Don Kuenz KB7RPU
>
>Science is built up with facts, as a house is with stones. But a
>collection of facts is no more a science than a heap of stones is a
>house. - Poincare

Which version of Python do you wish to use?
-- 
Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! 
http://www.fullyfollow.me/rootnl2k  Look at Psalms 14 and 53 on Atheism
Time for the USA to hold a referendum on its republic and vote to dissolve!! 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get the sum of differences between integers in a list

[Steve D'Aprano]

On Tue, 20 Sep 2016 09:52 pm, Daiyue Weng wrote:

> Hi, I have a list numbers say,
> 
> [1,2,3,4,6,8,9,10,11]
> 
> First, I want to calculate the sum of the differences between the numbers
> in the list.

Sounds like homework. What have you tried?


Hint: use the sum() function to sum a list of numbers:

py> sum([1, 2, 3])
6

Another hint: to calculate the differences between numbers, you need to look
at each number except the first, and subtract the *previous* number. Or
possibly the *next* number. You decide what the question wants you to do.
One way to do that is to loop over the INDEXES 0, 1, 2, 3, ... up to the
number of items:

for index in range(len(some_list)):
x = some_list[i]


A third hint (for advanced students): you can use zip() and list slicing to
get the numbers two at a time without using the indexes.



> Second, if a sequence of numbers having a difference of 1, put them in a
> list, i.e. there are two such lists,
> 
> [1,2,3]
> 
> [8,9,10,11]
> 
> and also put the rest numbers in another list, i.e. there is only one such
> list in the example,
> 
> [6].


Hint: for each number, decide whether or not the difference is 1. If the
difference is *not* one, that's easy: just place that number into a second
list.

If the difference is 1, then you place that number into a third list, BUT
only as part of a run. So you'll go:

difference between 1 and 2 is 1, so 1 goes into list B;
difference between 2 and 3 is 1, so 2 goes into list B;
difference between 3 and 4 is 1, so 3 goes into list B;
difference between 4 and 6 is 2, so 4 goes into list A;
difference between 6 and 8 is 2, so 6 goes into list A;
difference between 8 and 9 is 1, so list B is put aside, 
a new list is started, and 8 goes into this new list B;

etc.

Hint: try keeping a list of lists. To start a new list, append an empty list
to it.



> Third, get the lists with the max/min sizes from above, i.e. in this
> example, the max list is,
> 
> [8,9,10,11]
> 
> min list is
> 
> [1,2,3].


Hint: 

min([5, 6, 4, 1, 2, 3]) returns the smallest number, 1.
max([5, 6, 4, 1, 2, 3]) returns the largest number, 6.

Use len() to get the size of a list.



> What's the best way to implement this?

By writing code, of course! But writing code should be the SECOND thing you
do, not the first.

First, work through the calculation by hand. How do you do it by hand, using
pen and paper?

Now write some code to do the same thing!

Good luck!



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to install Python.h on FreeBSD 10.3-RELEASE?

[Don Kuenz]

It turns out that the question isn't "How to install Python.h?" The
question is "Why doesn't make find Python.h?"



# uname -v
FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014

# cd /
# find . -name 'Python.h'
/usr/local/include/python2.7/Python.h
/usr/local/include/python3.4m/Python.h



$ uname -v
FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016

# cd /
# find . -name 'Python.h'
/usr/local/include/python2.7/Python.h
/usr/local/include/python3.4m/Python.h



Making all in src/cpython
make  all-am
depbase=`echo data_arc.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`; /bin/sh
In file included from data_arc.c:19:0:
data.h:22:20: fatal error: Python.h: No such file or directory
 #include 
^
compilation terminated.
*** Error code 1



It seems that a symbolic link may be needed. Fortunately the developer
is working with me on this problem. He may know the best way to proceed.

Thank you,

--
Don Kuenz KB7RPU

Science is built up with facts, as a house is with stones. But a
collection of facts is no more a science than a heap of stones is a
house. - Poincare
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What does this zip() code mean?

[380162267qq]

在 2016年9月20日星期二 UTC-4上午9:35:50，Random832写道：
> On Tue, Sep 20, 2016, at 09:19, 38016226...@gmail.com wrote:
> > >>> x = [1, 2, 3]
> > >>> y = [4, 5, 6]
> > >>> zipped = zip(x, y)
> > >>> list(zipped)
> > [(1, 4), (2, 5), (3, 6)]
> > >>> x2, y2 = zip(*zip(x, y))
> > >>> x == list(x2) and y == list(y2)
> > True
> > 
> > My problem is >>> x2, y2 = zip(*zip(x, y)).
> > zip return an iterator but x2 and y2 are different?
> > I really need detail explanation about this line.
> 
> Well, as you've seen, zip(x, y) is (1, 4), (2, 5), (3, 6)
> 
> This means that zip(*...) is zip((1, 4), (2, 5), (3, 6)).  It takes the
> first element of each argument (1, 2, and 3), and then the next element
> of each argument (4, 5, and 6).

thank you!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What does this zip() code mean?

[Random832]

On Tue, Sep 20, 2016, at 09:19, 38016226...@gmail.com wrote:
> >>> x = [1, 2, 3]
> >>> y = [4, 5, 6]
> >>> zipped = zip(x, y)
> >>> list(zipped)
> [(1, 4), (2, 5), (3, 6)]
> >>> x2, y2 = zip(*zip(x, y))
> >>> x == list(x2) and y == list(y2)
> True
> 
> My problem is >>> x2, y2 = zip(*zip(x, y)).
> zip return an iterator but x2 and y2 are different?
> I really need detail explanation about this line.

Well, as you've seen, zip(x, y) is (1, 4), (2, 5), (3, 6)

This means that zip(*...) is zip((1, 4), (2, 5), (3, 6)).  It takes the
first element of each argument (1, 2, and 3), and then the next element
of each argument (4, 5, and 6).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[380162267qq]

在 2016年9月20日星期二 UTC-4上午9:13:35，Peter Otten写道：
> 38016226...@gmail.com wrote:
> 
> > 在 2016年9月20日星期二 UTC-4上午8:17:13，BartC写道：
> >> On 20/09/2016 13:12, 38016226...@gmail.com wrote:
> >>  d = {}
> >>  keys = range(256)
> >>  vals = map(chr, keys)
> >>  map(operator.setitem, [d]*len(keys), keys, vals)
> >> >
> >> > It is from python library. What does [d]*len(keys) mean?
> >> > d is the name of dict but put d in [] really confused me.
> >> >
> >> 
> >> if len(keys) is 5 then [d]*5 is:
> >> 
> >> [d,d,d,d,d]
> >> 
> >> [d] is a list, containing one item, a dict if that is what it is.
> >> 
> >> --
> >> Bartc
> > 
> > Thank you. I understand now
> 
> It should be noted that the code above is really bad Python.
> Better alternatives are the simple loop
> 
> d = {}
> for i in range(256):
>  d[i] = chr(i)
> 
> or the dict comprehension
> 
> d = {i: chr(i) for i in range(256)}
> 
> and even
> 
> keys = range(256)
> d = dict(zip(keys, map(chr, keys)))
> 
> because they don't build lists only to throw them away. 
> 
> 
> Also, creating a list of dicts or lists is a common gotcha because after
> 
> outer = [[]] * 3
> 
> the outer list contains *the* *same* list three times, not three empty 
> lists. Try
> 
> outer[0].append("surprise")
> print(outer)
> 
> in the interactive interpreter to see why the difference matters.
> 
> 
> Finally, if you are just starting you might consider picking Python 3 
> instead of Python 2.

Thank you.I learn more!
-- 
https://mail.python.org/mailman/listinfo/python-list

What does this zip() code mean?

[380162267qq]

>>> x = [1, 2, 3]
>>> y = [4, 5, 6]
>>> zipped = zip(x, y)
>>> list(zipped)
[(1, 4), (2, 5), (3, 6)]
>>> x2, y2 = zip(*zip(x, y))
>>> x == list(x2) and y == list(y2)
True

My problem is >>> x2, y2 = zip(*zip(x, y)).
zip return an iterator but x2 and y2 are different?
I really need detail explanation about this line.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[Peter Otten]

38016226...@gmail.com wrote:

> 在 2016年9月20日星期二 UTC-4上午8:17:13，BartC写道：
>> On 20/09/2016 13:12, 38016226...@gmail.com wrote:
>>  d = {}
>>  keys = range(256)
>>  vals = map(chr, keys)
>>  map(operator.setitem, [d]*len(keys), keys, vals)
>> >
>> > It is from python library. What does [d]*len(keys) mean?
>> > d is the name of dict but put d in [] really confused me.
>> >
>> 
>> if len(keys) is 5 then [d]*5 is:
>> 
>> [d,d,d,d,d]
>> 
>> [d] is a list, containing one item, a dict if that is what it is.
>> 
>> --
>> Bartc
> 
> Thank you. I understand now

It should be noted that the code above is really bad Python.
Better alternatives are the simple loop

d = {}
for i in range(256):
 d[i] = chr(i)

or the dict comprehension

d = {i: chr(i) for i in range(256)}

and even

keys = range(256)
d = dict(zip(keys, map(chr, keys)))

because they don't build lists only to throw them away. 


Also, creating a list of dicts or lists is a common gotcha because after

outer = [[]] * 3

the outer list contains *the* *same* list three times, not three empty 
lists. Try

outer[0].append("surprise")
print(outer)

in the interactive interpreter to see why the difference matters.


Finally, if you are just starting you might consider picking Python 3 
instead of Python 2.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[380162267qq]

在 2016年9月20日星期二 UTC-4上午8:17:13，BartC写道：
> On 20/09/2016 13:12, 38016226...@gmail.com wrote:
>  d = {}
>  keys = range(256)
>  vals = map(chr, keys)
>  map(operator.setitem, [d]*len(keys), keys, vals)
> >
> > It is from python library. What does [d]*len(keys) mean?
> > d is the name of dict but put d in [] really confused me.
> >
> 
> if len(keys) is 5 then [d]*5 is:
> 
> [d,d,d,d,d]
> 
> [d] is a list, containing one item, a dict if that is what it is.
> 
> -- 
> Bartc

Thank you. I understand now
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I am newbie who can explain this code to me?

[BartC]

On 20/09/2016 13:12, 38016226...@gmail.com wrote:

d = {}
keys = range(256)
vals = map(chr, keys)
map(operator.setitem, [d]*len(keys), keys, vals)


It is from python library. What does [d]*len(keys) mean?
d is the name of dict but put d in [] really confused me.



if len(keys) is 5 then [d]*5 is:

   [d,d,d,d,d]

[d] is a list, containing one item, a dict if that is what it is.

--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

I am newbie who can explain this code to me?

[380162267qq]

>>> d = {}
>>> keys = range(256)
>>> vals = map(chr, keys)
>>> map(operator.setitem, [d]*len(keys), keys, vals)

It is from python library. What does [d]*len(keys) mean?
d is the name of dict but put d in [] really confused me.
-- 
https://mail.python.org/mailman/listinfo/python-list

get the sum of differences between integers in a list

[Daiyue Weng]

Hi, I have a list numbers say,

[1,2,3,4,6,8,9,10,11]

First, I want to calculate the sum of the differences between the numbers
in the list.
Second, if a sequence of numbers having a difference of 1, put them in a
list, i.e. there are two such lists,

[1,2,3]

[8,9,10,11]

and also put the rest numbers in another list, i.e. there is only one such
list in the example,

[6].

Third, get the lists with the max/min sizes from above, i.e. in this
example, the max list is,

[8,9,10,11]

min list is

[1,2,3].

What's the best way to implement this?

cheers
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cython taking more time than regular Python

[Peter Otten]

Christian Gollwitzer wrote:

> Am 20.09.16 um 09:44 schrieb Peter Otten:
>> Stefan Behnel wrote:
>>
>>> Peter Otten schrieb am 19.09.2016 um 14:55:
 In [7]: %%cython
 def omega(int n):
 cdef long i
 cdef long result = 0
 for i in range(n): result += i
 return result
...:

 In [8]: %timeit omega(10)
 1 loops, best of 3: 91.6 µs per loop
>>>
>>> Note that this is the worst benchmark ever. Any non-dump C compiler will
>>> happily apply Young Gauß and calculate the result in constant time.
>>
>> Is that optimization useful in an actual program or just a way to cheat
>> in benchmarks?
> 
> Good question, but I think this falls under "strength reduction" which
> is a useful optimization of tight loops in C.
> 
> However, I'm not convinced it did succeed here. An evaluation of the
> Gauß formula would run in a few *nanoseconds* on any moddern machine. It
> may take a microsecond to call a functino from Python. a hundred
> microseconds means that the loop does run.

A quick check indeed shows linear behaviour:

In [1]: %load_ext cythonmagic

In [2]: %%cython
def sigma(int n):
cdef long i
cdef long result = 0
for i in range(n): result += i
return result
   ...: 

In [3]: %timeit sigma(1000)
100 loops, best of 3: 1.04 µs per loop

In [4]: %timeit sigma(1)
10 loops, best of 3: 9.25 µs per loop

In [5]: %timeit sigma(10)
1 loops, best of 3: 91.3 µs per loop


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cython taking more time than regular Python

[Christian Gollwitzer]

Am 20.09.16 um 09:44 schrieb Peter Otten:

Stefan Behnel wrote:


Peter Otten schrieb am 19.09.2016 um 14:55:

In [7]: %%cython
def omega(int n):
cdef long i
cdef long result = 0
for i in range(n): result += i
return result
   ...:

In [8]: %timeit omega(10)
1 loops, best of 3: 91.6 µs per loop


Note that this is the worst benchmark ever. Any non-dump C compiler will
happily apply Young Gauß and calculate the result in constant time.


Is that optimization useful in an actual program or just a way to cheat in
benchmarks?


Good question, but I think this falls under "strength reduction" which 
is a useful optimization of tight loops in C.


However, I'm not convinced it did succeed here. An evaluation of the 
Gauß formula would run in a few *nanoseconds* on any moddern machine. It 
may take a microsecond to call a functino from Python. a hundred 
microseconds means that the loop does run.


Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: Cython taking more time than regular Python

[BartC]

On 20/09/2016 06:27, Stefan Behnel wrote:

Peter Otten schrieb am 19.09.2016 um 14:55:

In [7]: %%cython
def omega(int n):
cdef long i
cdef long result = 0
for i in range(n): result += i
return result
   ...:

In [8]: %timeit omega(10)
1 loops, best of 3: 91.6 µs per loop


Note that this is the worst benchmark ever. Any non-dump C compiler will
happily apply Young Gauß and calculate the result in constant time.


Which 'non-dump' C compiler will actually do such an optimisation? (Just 
so I can steer clear of it; I don't want a compiler wasting /my/ time 
doing pointless analysis.)


This actually quite a good micro-benchmark precisely because that would 
be an unusual optimisation.


(With benchmarking you usually want to know how long it takes a program 
to perform a task, not how long it takes to *not* do it!)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module

[Chris Angelico]

On Tue, Sep 20, 2016 at 4:19 PM, dl l  wrote:
> Yes, it's a workaround to set the global variables to None. But My app just
> provide a framework for my customers to run python scripts. That's means
> the workaround requires my customers to update their python scripts. That
> may make them unhappy :(. Is there a workaround in my code C++ side to call
> Python C APIs to resolve the memory leak issue?
>

Python's import system is fundamentally not designed for what I
believe is your use-case here, of running potentially-edited user
code. Instead, I suggest keeping 'import' for stable modules that
don't get updated without restarting the process, and for the more
dynamic code, build something using exec (or PyRun_StringFlags, which
I believe is the C equivalent). Object lifetimes will then be entirely
under your control.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cython taking more time than regular Python

[Peter Otten]

Stefan Behnel wrote:

> Peter Otten schrieb am 19.09.2016 um 14:55:
>> In [7]: %%cython
>> def omega(int n):
>> cdef long i
>> cdef long result = 0
>> for i in range(n): result += i
>> return result
>>...:
>> 
>> In [8]: %timeit omega(10)
>> 1 loops, best of 3: 91.6 µs per loop
> 
> Note that this is the worst benchmark ever. Any non-dump C compiler will
> happily apply Young Gauß and calculate the result in constant time.

Is that optimization useful in an actual program or just a way to cheat in 
benchmarks?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module

[Steven D'Aprano]

On Tuesday 20 September 2016 16:19, dl l wrote:

> Hi Steven,
> 
> Thanks for reply.
> 
> I logged bug https://bugs.python.org/issue28202. I added more info in this
> bug :).
> 
> Yes, it's a workaround to set the global variables to None. But My app just
> provide a framework for my customers to run python scripts. That's means
> the workaround requires my customers to update their python scripts. That
> may make them unhappy :(. Is there a workaround in my code C++ side to call
> Python C APIs to resolve the memory leak issue?

Sorry, I wasn't clear. I meant to say that from your C++ side you should try 
explicitly setting the object to None to see if the __del__ method runs.

I don't know C++, which is why I wrote it in Python.


Assuming that this really is a memory leak, then you could try walking the 
module and deleting everything by hand. Again, I'm writing this in Python, but 
you should write it in C++.

# instead of just this:
del sys.modules['mymodule']  # PyDict_DelItemString


# try this instead
mod = sys.modules['mymodule']
namespace = vars(mod)
for name in list(namespace.keys()):
del namespace[name]  # PyDict_DelItemString ?
del mod
del sys.modules['mymodule']  # PyDict_DelItemString


But it might be easiest to just update once the bug is fixed.




-- 
Steven
git gets easier once you get the basic idea that branches are homeomorphic 
endofunctors mapping submanifolds of a Hilbert space.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Another å, ä, ö question

[Steven D'Aprano]

On Tuesday 20 September 2016 06:21, Martin Schöön wrote:

> I am puzzled: With Geany there is no problem but outside Geany
> I am punished by Python for using Swedish.
> 
> Any ideas?

Yes, you are being punished for what Sweden did to Julian Assange.

No, only kidding.

We can't really help you if you don't tell us exactly what error you get. Is 
this an error in emacs? A Python error? What exception do you get?

But if I were to take a guess, I think you should put an encoding cookie in the 
first or second line of your text file. Make sure your editor is configured to 
use UTF-8 as the encoding, and put this in line 1 or line 2 of the file:


# -*- coding: utf-8 -*-


That will tell Python to read your file as UTF-8 instead of ASCII.



-- 
Steven
git gets easier once you get the basic idea that branches are homeomorphic 
endofunctors mapping submanifolds of a Hilbert space.

-- 
https://mail.python.org/mailman/listinfo/python-list