from:"Dave"


On 05/09/2015 07:01 PM, Vincent Davis wrote:

Not sure what I was doing wrong, it seems to work now.



I still see two significant things wrong:

1) you're top-posting, putting your response  BEFORE the stuff you're 
responding to.


2) both messages are in html, which thoroughly messed up parts of your 
error messages.




On Sat, May 9, 2015 at 4:46 PM, Vincent Davis 
wrote:


I am reading a file with Dictreader and writing a new file. I want use the
fieldnames in the Dictwriter from the reader. See below How should I be
doing this?



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?


On 05/09/2015 05:04 PM, vjp2...@at.biostrategist.dot.dot.com wrote:

Thanks.. I suspected it wasn't meant to be taken as in the file

THe one thing I'm not sure if Jython is suppsosedto keep running
after the initisl stuff is loaded in..


To put the question in purely DOS terms if you run a program can you pipe it
some commands and then keep it running to take the remaining commands from
the console?



That's not a built-in feature of cmd.exe.  However, it wouldn't be hard 
to write a data source (funny.exe) that took data from a file, and then 
from stdin, sending both in progression to stdout.  Then you'd run the 
two programs as:


funny.exe infile.txt | newprog.exe


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape


On 05/09/2015 06:31 AM, zljubisic...@gmail.com wrote:


title = title[:232]
title = title.replace(" ", "_").replace("/", "_").replace("!", "_").replace("?", 
"_")\
 .replace('"', "_").replace(':', "_").replace(',', 
"_").replace('"', '')\
 .replace('\n', '_').replace(''', '')

print(title)

src_file = os.path.join(ROOTDIR, 'src_' + title + '.txt')
dst_file = os.path.join(ROOTDIR, 'des_' + title + '.txt')

print(len(src_file), src_file)
print(len(dst_file), dst_file)

with open(src_file, mode='w', encoding='utf-8') as s_file:
 s_file.write('test')


shutil.move(src_file, dst_file)

It works, but if you change title = title[:232] to title = title[:233], you will get 
"FileNotFoundError: [Errno 2] No such file or directory".
As you can see ROOTDIR contains \U.


No, we can't see what ROOTDIR is, since you read it from the config 
file.  And you don't show us the results of those prints.  You don't 
even show us the full exception, or even the line it fails on.


I doubt that the problem is in the ROODIR value, but of course nothing 
in your program bothers to check that that directory exists.  I expect 
you either have too many characters total, or the 232th character is a 
strange one.  Or perhaps title has a backslash in it (you took care of 
forward slash).


While we're at it, if you do have an OS limitation on size, your code is 
truncating at the wrong point.  You need to truncate the title based on 
the total size of src_file and dst_file, and since the code cannot know 
the size of ROOTDIR, you need to include that in your figuring.





--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: calling base class method fetches no results


On 05/09/2015 03:59 AM, david jhon wrote:

Hi, I am sorry for sending in five attachments, I cloned the code from here
: Let me explain it here:



Please don't top-post.  Your earlier problem description, which I could 
make no sense of, is now located after your later "clarification".


Thanks for eliminating the attachments.  Many cannot see them.  And for 
extracting only part of the code into the message.  It's still too much 
for me, but others may manage it okay.  To me, it seems likely that most 
of that code will not have any part in the problem you're describing. 
And in some places you have code that's missing its context.


Now, eliminate the pieces of code that are irrelevant to your question, 
and state the problem in terms that make sense.  Somebody is 
instantiating an object.  Exactly which class is being used for that? 
Somebody else is calling a particular method (be specific, rather than 
just saying "some method"), and it's giving the wrong results.  And 
apparently, those wrong results depend on which source file something 
happens in.




Routing Base class defined in DCRouting.py:

import logging
from copy import copy

class Routing(object):
 '''Base class for data center network routing.

 Routing engines must implement the get_route() method.
 '''

 def __init__(self, topo):
 '''Create Routing object.

 @param topo Topo object from Net parent
 '''
 self.topo = topo

 def get_route(self, src, dst, hash_):
 '''Return flow path.

 @param src source host
 @param dst destination host
 @param hash_ hash value

 @return flow_path list of DPIDs to traverse (including hosts)
 '''
 raise NotImplementedError

 def routes(self, src, dst):
 ''' Return list of paths

 Only works for Fat-Tree topology

 @ param src source host
 @ param dst destination host

 @ return list of DPIDs (including inputs)
 '''

 complete_paths = [] # List of complete dpid routes

 src_paths = { src : [[src]] }
 dst_paths = { dst : [[dst]] }

 dst_layer = self.topo.layer(dst)
 src_layer = self.topo.layer(src)

 lower_layer = src_layer
 if dst_layer > src_layer:
 lower_layer = dst_layer


 for front_layer in range(lower_layer-1, -1, -1):
 if src_layer > front_layer:
 # expand src frontier
 new_src_paths = {}
 for node in sorted(src_paths):
 path_list = src_paths[node]
 for path in path_list:
 last_node = path[-1]
 for frontier_node in
self.topo.upper_nodes(last_node):
 new_src_paths[frontier_node] = [path +
[frontier_node]]

 if frontier_node in dst_paths:
 dst_path_list = dst_paths[frontier_node]
 for dst_path in dst_path_list:
 dst_path_copy = copy ( dst_path )
 dst_path_copy.reverse()
 complete_paths.append( path +
dst_path_copy)
 src_paths = new_src_paths

 if dst_layer > front_layer:
 # expand dst frontier
 new_dst_paths = {}
 for node in sorted(dst_paths):
 path_list = dst_paths[node]
 for path in path_list:
 last_node = path[-1]
 for frontier_node in
self.topo.upper_nodes(last_node):
 new_dst_paths[frontier_node] = [ path +
[frontier_node]]

 if frontier_node in src_paths:
 src_path_list = src_paths[frontier_node]
 dst_path_copy = copy( path )
 dst_path_copy.reverse()
 for src_path in src_path_list:
 complete_paths.append( src_path +
dst_path_copy)

 dst_paths = new_dst_paths

 if complete_paths:
 return complete_paths
class HashedRouting(Routing):
 ''' Hashed routing '''

 def __init__(self, topo):
 self.topo = topo

 def get_route(self, src, dst, hash_):
 ''' Return flow path. '''

 if src == dst:
 return [src]

 paths = self.routes(src,dst)
 if paths:
 #print 'hash_:', hash_
 choice = hash_ % len(paths)
 #print 'choice:', choice
 path = sorted(paths)[choice]
 #print 'path:', path
 return path

>
Instantiated in util.py:

from DCTopo import FatTreeTopo
from mininet.util import makeNumeric
from DCRouting import H

Re: Encrypt python files

2015-05-08 Thread Dave Angel


On 05/08/2015 06:59 AM, Denis McMahon wrote:

On Wed, 06 May 2015 00:23:39 -0700, Palpandi wrote:


On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:

Hi,

What are the ways to encrypt python files?


No, I just want to hide the scripts from others.


You can do that by deleting the scripts. Make sure you use a secure
deletion tool.

I'm not aware of any mechanism for encrypted executable python scripts.
You can obfuscate the code, but you can't encrypt it because the python
interpreter needs the script file to execute it.

The same holds true for any mechanism designed to encrypt executable code
regardless of whether it's script or compiled. At the lowest level the
processor only understands the instruction set, and encrypted code has to
be decrypted to execute.

As the decryption method is always available to anyone who has legitimate
access to execute the code, it's impossible to hide the code at that
point.

Example - if I give you an encrypted binary to run on your system, it
either has to be unencryptable


It'd be clearer if you used  decryptable, since unencryptable has a very 
different meaning.


http://en.wiktionary.org/wiki/unencryptable



using tools you already have, or using a
built in unencrypter, both of which you have access to and can use to
unencrypt the encrypted executable code.



likewise decrypter and decrypt.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is this unpythonic?

2015-05-08 Thread Dave Angel


On 05/08/2015 06:53 AM, Frank Millman wrote:


"Steven D'Aprano"  wrote in message
news:554c8b0a$0$12992$c3e8da3$54964...@news.astraweb.com...

On Fri, 8 May 2015 06:01 pm, Frank Millman wrote:


Hi all


[...]


However, every time I look at my own code, and I see   "def x(y, z=[]):
."   it looks wrong because I have been conditioned to think of it as
a gotcha.




It might be appropriate to define the list at top-level, as

EMPTY_LIST=[]

and in your default argument as
def x(y, z=EMPTY_LIST):

and with the all-caps, you're thereby promising that nobody will modify 
that list.


(I'd tend to do the None trick, but I think this alternative would be 
acceptable)


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

2015-05-08 Thread Dave Angel


On 05/08/2015 02:42 AM, Chris Angelico wrote:

On Fri, May 8, 2015 at 4:36 PM, Rustom Mody  wrote:

On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:

Why have the concept of a procedure?


On Friday, Chris Angelico ALSO wrote:

With print(), you have a conceptual procedure...


So which do you want to stand by?


A procedure, in Python, is simply a function which returns None.
That's all. It's not any sort of special concept. It doesn't need to
be taught. If your students are getting confused by it, stop teaching
it!


One thing newbies get tripped up by is having some path through their 
code that doesn't explicitly return.  And in Python that path therefore 
returns None.  It's most commonly confusing when there are nested ifs, 
and one of the "inner ifs" doesn't have an else clause.


Anyway, it's marginally more useful to that newbie if the compiler would 
produce an error instead of later seeing a runtime error due to an 
unexpected None result.


I don't think Python would be improved by detecting such a condition and 
reporting on it.  That's a job for a linter, or a style guide program.


No different than the compile time checks for variable type that most 
languages impose.  They don't belong in Python.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: PEP idea: On Windows, subprocess should implicitly support .bat and .cmd scripts by using FindExecutable from win32 API

2015-05-07 Thread Dave Angel


On 05/07/2015 06:24 AM, Chris Angelico wrote:

On Thu, May 7, 2015 at 8:10 PM, Marko Rauhamaa  wrote:

Stefan Zimmermann :


And last but not least, Popen behavior on Windows makes it difficult
to write OS-independent Python code which calls external commands that
are not binary by default:


Then, write OS-dependent Python code.

I don't think it's Python's job to pave over OS differences. Java does
that by not offering precious system facilities -- very painful. Python
is taking steps in that direction, but I hope it won't go too far.


On the contrary, I think it *is* a high level language's job to pave
over those differences. Portable C code generally has to have a
whopping 'configure' script that digs into your hardware, OS, library,
etc availabilities, and lets you figure out which way to do things.
Python code shouldn't need to worry about that. You don't need to care
whether you're on a 32-bit or 64-bit computer; you don't need to care
whether it's an Intel chip or a RISCy one; you shouldn't have to
concern yourself with the difference between BSD networking and
WinSock. There'll be a handful of times when you do care, and for
those, it's nice to have some facilities exposed; but the bulk of code
shouldn't need to know about the platform it's running on.

Java went for a philosophy of "write once, run anywhere" in its early
days, and while that hasn't exactly been stuck to completely, it's
still the reasoning behind the omission of certain system facilities.
Python accepts and understands that there will be differences, so you
can't call os.getuid() on Windows, and there are a few restrictions on
the subprocess module if you want maximum portability, but the bulk of
your code won't be any different on Linux, Windows, Mac OS, OS/2,
Amiga, OS/400, Solaris, or a MicroPython board.

ChrisA



It's a nice goal.  But these aren't OS features in Windows, they're 
shell features.  And there are several shells.  If the user has 
installed a different shell, is it Python's job to ignore it and 
simulate what cmd.exe does?


Seems to me that's what shell=True is for.  it signals Python that we're 
willing to trust the shell to do whatever magic it chooses, from adding 
extensions, to calling interpreters, to changing search order, to 
parsing the line in strange ways, to setting up temporary environment 
contexts, etc.


If there were just one shell, it might make sense to emulate its 
features.  Or it might make sense to contort its features to look like a 
Unix shell.  But with multiple possibilities, seems that's more like 
space for a 3rd party library.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: PEP idea: On Windows, subprocess should implicitly support .bat and .cmd scripts by using FindExecutable from win32 API


On 05/06/2015 06:11 PM, Stefan Zimmermann wrote:

Hi.

I don't like that subprocess.Popen(['command']) only works on Windows if there 
is a command.exe in %PATH%.


 As a Windows user you would normally expect that also command.bat and 
command.cmd can be run that way.




and command.com.

If it's really unfortunate that you picked "command" for your sample 
program name.  Since command.com was the shell in MSDOS, I was about to 
point you to COMSPEC to address your problem.


There's nothing Windows-specific about that behaviour.  In Linux, there 
are bash commands that can only be run by using shell=True.  Fortunately 
Popen didn't make the mistake of pretending it's a shell.


There is lots more to running a batch file than launching it.  The whole 
syntax of the rest of the commandline differs when you're doing that.



There are simple workarounds like Popen(..., shell=True) but that is a heavy 
overhead for .exe files.


And the reason there's such an overhead is because you're requesting the 
services of the shell.  If you don't need those services, use shell=False.




Currently I use pywin32 and call Popen([win32api.FindExecutable('command')[1]]) 
as a workaround. This has zero overhead.

It should be default for Popen to call FindExecutable internally.

Was this discussed before?
Is it worth a PEP?
Or at least an issue?

Cheers,
Stefan




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: extracting zip item to in-memory


On 05/06/2015 04:27 PM, noydb wrote:

I have a zip file containing several files and I want to extract out just the 
.xml file.  I have that code.  Curious if this xml file can be extracted into 
memory.  If so, how to?  I only need to move the file around, and maybe read 
some tags.

Thanks for any help!

python 2.7



See https://docs.python.org/2.7/library/zipfile.html#zipfile.ZipFile.open

To open a particular member and get back a file-like object.

Once you have that, you can use the  read() method of that object.

Once you've coded this, if it doesn't work, post what you've got with a 
description of what doesn't work, and somebody here will be able to fix 
it up.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/06/2015 11:36 AM, Alain Ketterlin wrote:

Yes, plus the time for memory allocation. Since the code uses "r *=
...", space is reallocated when the result doesn't fit. The new size is
probably proportional to the current (insufficient) size. This means
that overall, you'll need fewer reallocations, because allocations are
made in bigger chunks.


That sounds plausible, but  a=5; a*=4  does not update in place. It 
calculates and creates a new object.  Updating lists can work as you 
say, but an int is immutable.


It's an optimization that might be applied if the code generator were a 
lot smarter, (and if the ref count is exactly 1), but it would then be 
confusing to anyone who used id().


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/06/2015 09:55 AM, Chris Angelico wrote:

On Wed, May 6, 2015 at 11:12 PM, Dave Angel  wrote:

I had guessed that the order of multiplication would make a big difference,
once the product started getting bigger than the machine word size.

Reason I thought that is that if you multiply starting at the top value (and
end with multiplying by 2) you're spending more of the time multiplying
big-ints.

That's why I made sure that both Cecil's and my implementations were
counting up, so that wouldn't be a distinction.

I'm still puzzled, as it seems your results imply that big-int*int is faster
than int*int where the product is also int.


Are you using Python 2 or Python 3 for your testing? In Py3, there's
no type difference, and barely no performance difference as you cross
the word-size boundary. (Bigger numbers are a bit slower to work with,
but not hugely.)



Both Cecil and I are using 3.x  I'm using 3.4 in particular.  And I know 
int covers both big-int and int32.  that's why I called it big-int, 
rather than long.


I was, however, mistaken.  it's not that threshold that we're crossing 
here, but another one, for MUCH larger numbers.  factorial of 10 and 
of 20 have 456473 and 97350 digits, respectively.  In binary, that 
would be about 190k bytes and 404k bytes, respectively.


I was seeing factorial of 20 taking about 4.5 times as long as 
factorial of 10.  All the other increments seemed fairly proportional.


I'll bet the difference is something like the memory allocator using a 
different algorithm for blocks above 256k.  Or the cache logic hitting a 
threshold.


If it's caching, of course the threshold will differ wildly between 
machine architectures.


If it's the memory allocator, that could easily vary between Python 
versions as well.





--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/06/2015 02:26 AM, Steven D'Aprano wrote:

On Wednesday 06 May 2015 14:05, Steven D'Aprano wrote:



My interpretation of this is that the difference has something to do with
the cost of multiplications. Multiplying upwards seems to be more expensive
than multiplying downwards, a result I never would have predicted, but
that's what I'm seeing. I can only guess that it has something to do with
the way multiplication is implemented, or perhaps the memory management
involved, or something. Who the hell knows?



I had guessed that the order of multiplication would make a big 
difference, once the product started getting bigger than the machine 
word size.


Reason I thought that is that if you multiply starting at the top value 
(and end with multiplying by 2) you're spending more of the time 
multiplying big-ints.


That's why I made sure that both Cecil's and my implementations were 
counting up, so that wouldn't be a distinction.


I'm still puzzled, as it seems your results imply that big-int*int is 
faster than int*int where the product is also int.


That could use some more testing, though.

I still say a cutoff of about 10% is where we should draw the line in an 
interpretive system.  Below that, you're frequently measuring noise and 
coincidence.


Remember the days when you knew how many cycles each assembly 
instruction took, and could simply add them up to compare algorithms?



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 05:39 PM, Ian Kelly wrote:

On Tue, May 5, 2015 at 3:23 PM, Ian Kelly  wrote:

On Tue, May 5, 2015 at 3:00 PM, Dave Angel  wrote:

def loop(func, funcname, arg):
 start = time.time()
 for i in range(repeats):
 func(arg, True)
 print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

 start = time.time()
 for i in range(repeats):
 func(arg)
 print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))


Note that you're explicitly passing True in one case but leaving the
default in the other. I don't know whether that might be responsible
for the difference you're seeing.


I don't think that's the cause, but I do think that it has something
to do with the way the timing is being run. When I run your loop
function, I do observe the difference. If I reverse the order so that
the False case is tested first, I observe the opposite. That is, the
slower case is consistently the one that is timed *first* in the loop
function, regardless of which case that is.



I created two functions and called them with Timeit(), and the 
difference is now below 3%


And when I take your lead and double the loop() function so it runs each 
test twice, I get steadily decreasing numbers.


I think all of this has been noise caused by the caching of objects 
including function objects.  I was surprised by this, as the loops are 
small enough I'd figure the function object would be fully cached the 
first time it was called.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 04:30 PM, Ian Kelly wrote:

On Tue, May 5, 2015 at 12:45 PM, Dave Angel  wrote:

When the "simple" is True, the function takes noticeably and consistently
longer.  For example, it might take 116 instead of 109 seconds.  For the
same counts, your code took 111.


I can't replicate this. What version of Python is it, and what value
of x are you testing with?


I've looked at dis.dis(factorial_iterative), and can see no explicit reason
for the difference.


My first thought is that maybe it's a result of the branch. Have you
tried swapping the branches, or reimplementing as separate functions
and comparing?



Logic is quite simple:


def factorial_iterative(x, simple=False):
assert x >= 0
result = 1
j=2
if not simple:
for i in range(2, x + 1):
#print("range value is of type", type(i), "and value", i)
#print("ordinary value is of type", type(j), "and value", j)
result *= i
j += 1
else:
for i in range(2, x + 1):
result *= j
j += 1

return result

def loop(func, funcname, arg):
start = time.time()
for i in range(repeats):
func(arg, True)
print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

start = time.time()
for i in range(repeats):
func(arg)
print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

repeats = 1

and arg is 10**4
loop(factorial_iterative,  "factorial_iterative  ", arg)

My actual program does the same thing with other versions of the 
function, including Cecil's factorial_tail_recursion, and my optimized 
version of that.



Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux

factorial_iterative  (10) took   3.807
factorial_iterative  (10) took   3.664

factorial_iterative  (20) took   17.07
factorial_iterative  (20) took15.3

factorial_iterative  (30) took   38.93
factorial_iterative  (30) took   36.01


Note that I test them in the opposite order of where they appear in the 
function.  That's because I was originally using the simple flag to test 
an empty loop.  The empty loop is much quicker either way, so it's not 
the issue.  (But if it were, the for-range version is much quicker).


I think I'll take your last suggestion and write separate functions.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string


On 05/05/2015 02:19 PM, Paul Moore wrote:

You need to specify that you're using Python 3.4 (or whichever) when 
starting a new thread.



I want to write a string to an already-open file (sys.stdout, typically). 
However, I *don't* want encoding errors, and the string could be arbitrary 
Unicode (in theory). The best way I've found is

 data = data.encode(file.encoding, errors='replace').decode(file.encoding)
 file.write(data)

(I'd probably use backslashreplace rather than replace, but that's a minor 
point).

Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it 
was the best I could think of.

Thanks,
Paul.



If you're going to take charge of the encoding of the file, why not just 
open the file in binary, and do it all with

file.write(data.encode( myencoding, errors='replace') )

i can't see the benefit of two encodes and a decode just to write a 
string to the file.


Alternatively, there's probably a way to open the file using 
codecs.open(), and reassign it to sys.stdout.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 12:18 PM, Cecil Westerhof wrote:



Well, I did not write many tail recursive functions. But what surprised
me was that for large values the ‘tail recursive’ version was more
efficient as the iterative version. And that was with myself
implementing the tail recursion. I expect the code to be more
efficient when the compiler implements the tail recursion.




You've said that repeatedly, so I finally took a look at your webpage

https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

I didn't have your framework to call the code, so I just extracted some 
functions and did some testing.  I do see some differences, where the 
so-called tail_recursive functions are sometimes faster, but I did some 
investigating to try to determine why.



I came up with the conclusion that sometimes the multiply operation 
takes longer than other times.  And in particular, i can see more 
variation between the two following loops than between your two functions.



def factorial_iterative(x, simple=False):
assert x >= 0
result = 1
j=2
if not simple:
for i in range(2, x + 1):
result *= i
j += 1
else:
for i in range(2, x + 1):
result *= j
j += 1
pass

return result

When the "simple" is True, the function takes noticeably and 
consistently longer.  For example, it might take 116 instead of 109 
seconds.  For the same counts, your code took 111.


I've looked at dis.dis(factorial_iterative), and can see no explicit 
reason for the difference.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages


On 05/05/2015 11:25 AM, Cecil Westerhof wrote:



I have a file with quotes and a file with tips. I want to place random
messages from those two (without them being repeated to soon) on my
Twitter page. This I do with ‘get_random_message’. I also want to put
the first message of another file and remove it from the file. For
this I use ‘dequeue_message’.



Removing lines from the start of a file is an n-squared operation. 
Sometiomes it pays to reverse the file once, and just remove from the 
end.  Truncating a file doesn't require the whole thing to be rewritten, 
nor risk losing the file if the make-new-file-rename-delete-old isn't 
quite done right.


Alternatively, you could overwrite the line, or more especially the 
linefeed before it.  Then you always do two readline() calls, using the 
second one's result.


Various other games might include storing an offset at the begin of 
file, so you start by reading that, doing a seek to the place you want, 
and then reading the new line from there.



Not recommending any of these, just bringing up alternatives.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bitten by my C/Java experience

2015-05-04 Thread Dave Angel


On 05/04/2015 04:28 PM, Cecil Westerhof wrote:

Op Monday 4 May 2015 21:39 CEST schreef Ian Kelly:


On Mon, May 4, 2015 at 11:59 AM, Mark Lawrence  wrote:

On 04/05/2015 16:20, Cecil Westerhof wrote:


Potential dangerous bug introduced by programming in Python as if
it was C/Java. :-( I used: ++tries that has to be: tries += 1

Are there other things I have to be careful on? That does not work
as in C/Java, but is correct syntax.



Not dangerous at all, your test code picks it up. I'd also guess,
but don't actually know, that one of the various linter tools could
be configured to find this problem.


pylint reports it as an error.


I installed it. Get a lot of messages. Mostly convention. For example:
 Unnecessary parens after 'print' keyword


Sounds like it's configured for Python 2.x.  There's probably a setting 
to tell it to use Python3 rules.




And:
 Invalid variable name "f"
for:
 with open(real_file, 'r') as f:


Sounds like a bad wording.  Nothing invalid about it, though it is a bit 
short.  There are certain one letter variables which are so common as to 
be expected, but others should be avoided.





But still something to add to my toolbox.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Converting 5.223701009526849e-05 to 5e-05

2015-05-03 Thread Dave Angel


On 05/03/2015 05:22 AM, Cecil Westerhof wrote:

Op Sunday 3 May 2015 10:40 CEST schreef Ben Finney:


Cecil Westerhof  writes:


When I have a value like 5.223701009526849e-05 in most cases I am
not interested in all the digest after the dot.


What type of value is it?


If the absolute value is bigger as 0 and smaller as 1, it should be a
float. ;-)



or for many uses, more likely a decimal.Decimal(), where many of the 
problems Ben mentions are moot.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Custom alphabetical sort

2015-05-02 Thread Dave Angel


On 05/02/2015 11:35 AM, Pander Musubi wrote:

On Monday, 24 December 2012 16:32:56 UTC+1, Pander Musubi  wrote:

Hi all,

I would like to sort according to this order:

(' ', '.', '\'', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 
'A', 'ä', 'Ä', 'á', 'Á', 'â', 'Â', 'à', 'À', 'å', 'Å', 'b', 'B', 'c', 'C', 'ç', 
'Ç', 'd', 'D', 'e', 'E', 'ë', 'Ë', 'é', 'É', 'ê', 'Ê', 'è', 'È', 'f', 'F', 'g', 
'G', 'h', 'H', 'i', 'I', 'ï', 'Ï', 'í', 'Í', 'î', 'Î', 'ì', 'Ì', 'j', 'J', 'k', 
'K', 'l', 'L', 'm', 'M', 'n', 'ñ', 'N', 'Ñ', 'o', 'O', 'ö', 'Ö', 'ó', 'Ó', 'ô', 
'Ô', 'ò', 'Ò', 'ø', 'Ø', 'p', 'P', 'q', 'Q', 'r', 'R', 's', 'S', 't', 'T', 'u', 
'U', 'ü', 'Ü', 'ú', 'Ú', 'û', 'Û', 'ù', 'Ù', 'v', 'V', 'w', 'W', 'x', 'X', 'y', 
'Y', 'z', 'Z')

How can I do this? The default sorted() does not give the desired result.

Thanks,

Pander


Meanwhile Python 3 supports locale aware sorting, see 
https://docs.python.org/3/howto/sorting.html



You're aware that the message you're responding to is 16 months old?

And answered pretty thoroughly, starting with the fact that the OP's 
desired order didn't match any particular locale.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python is not bad ;-)

2015-05-02 Thread Dave Angel


On 05/02/2015 05:33 AM, Cecil Westerhof wrote:

Please check your email settings.  Your messages that you type seem to 
be indented properly, but those that are quoting earlier messages (even 
your own) are not.  See below.  I suspect there's some problem with how 
your email program processes html messages.



Op Saturday 2 May 2015 10:26 CEST schreef Cecil Westerhof:


That is mostly because the tail recursion version starts multiplying
at the high end. I wrote a second version:
def factorial_tail_recursion2(x):
y = 1
z = 1
while True:
if x == z:
return y
y *= z
z += 1

This has almost the performance of the iterative version: 34 and 121
seconds.

So I made a new recursive version:
def factorial_recursive(x, y = 1, z = 1):
return y if x == z else factorial_recursive(x, x * y, z + 1)


Stupid me 'x == z' should be 'z > x'



I can't see how that is worth doing. The recursive version is already a 
distortion of the definition of factorial that I learned.  And to force 
it to be recursive and also contort it so it does the operations in the 
same order as the iterative version, just to gain performance?


If you want performance on factorial, write it iteratively, in as 
straightforward a way as possible.  Or just call the library function.


Recursion is a very useful tool in a developer's toolbox.  But the only 
reason I would use it for factorial is to provide a simple demonstration 
to introduce the concept to a beginner.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python is not bad ;-)

2015-05-02 Thread Dave Angel


On 05/02/2015 05:58 AM, Marko Rauhamaa wrote:

Chris Angelico :


Guido is against anything that disrupts tracebacks, and optimizing
tail recursion while maintaining traceback integrity is rather harder.


Tail recursion could be suppressed during debugging. Optimized code can
play all kinds of non-obvious tricks with the execution frame.


In the situations where it really is simple, you can always make the
change in your own code anyway. Often, the process of converting
recursion into tail recursion warps the code to the point where it's
abusing recursion to implement iteration anyway, so just make it
iterative.


While you shouldn't actively replace Python iteration with recursion, I
strongly disagree that naturally occurring tail recursion is abuse or
should be avoided in any manner.



When you strongly disagree, make sure you're disagreeing with what Chris 
actually said.  The key phrase in his message was

"converting recursion into tail recursion"

NOT  "converting iteration into recursion"
and NOT "naturally occurring tail recursion"


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK


On 04/30/2015 07:31 PM, Jon Ribbens wrote:

On 2015-04-30, Dave Angel  wrote:

Finally, I did some testing on Jon Ribben's version.  His was
substantially faster for smaller sets, and about the same for 10*7.  So
it's likely it'll be slower than yours and mine for 10**8.


You know what they say about assumptions. Actually, my version is six
times faster for 10**8 (running under Python 3.4).


But the real reason I didn't like it was it produced a much larger
set of happy_numbers, which could clog memory a lot sooner.  For
10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
had 1418854 happy members.


Er, what? You're complaining that mine is less efficient by not
producing the wrong output?



It's not intended as a criticism;  you solved a different problem.  The 
problem Cecil was solving was to determine if a particular number is 
happy.  The problem you solved was to make a list of all values under a 
particular limit that are happy.


Both produce identical results for the Cecil purpose, and yours is 
faster if one wants all the values.  But if one wants a sampling of 
values, his function will fetch them quickly, and even if you want them 
all, his function will use much less memory.


He keeps only one permutation of each value in the set, for substantial 
savings in space.  For example, he might just keep 28, while you keep 28 
and 82, 208, 280, 802, and 820.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Rounding a number


On 04/30/2015 06:35 PM, Seymore4Head wrote:

On Thu, 30 Apr 2015 22:00:17 +0200, Thijs Engels
 wrote:


round(65253, -3)

might be what you are looking for...


On Thu, Apr 30, 2015, at 21:49, Seymore4Head wrote:

I have this page book marked.
https://mkaz.com/2012/10/10/python-string-format/

I am getting numbers from sixty thousand to two hundred thousand.
I would like to round them to the nearest thousand.
So 65,253 should read 65,000.
How?

Total=2100
for x in range (10,35):
 count=1000/x
 print ("Foo {:7,.0f} Fighters".format(Total*count))
--
https://mail.python.org/mailman/listinfo/python-list


Thanks

I know there are more than one way to round and answer.  I was hoping
that using the {:7,.0f} formatting had a solution.



There are definite tradeoffs, but since you're rounding to integer size, 
the round() function works fine.  If you wanted tenths, you'd have to 
realize that a float (which is binary float) can't represent them 
exactly.  So depending on what further processing you do, you might see 
some effects that would not seem like it worked right.


Using the % or the .format style of formatting works fine, but it gives 
you a string.  You didn't specify that, so people probably assumed you 
wanted numbers.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK


On 04/30/2015 04:35 PM, Cecil Westerhof wrote:

Op Thursday 30 Apr 2015 20:53 CEST schreef Dave Angel:



Finally, I did some testing on Jon Ribben's version. His was
substantially faster for smaller sets, and about the same for 10*7.
So it's likely it'll be slower than yours and mine for 10**8. But
the real reason I didn't like it was it produced a much larger set
of happy_numbers, which could clog memory a lot sooner. For 10**7
items, I had 3250 happy members, and 19630 unhappy. And Jon had
1418854 happy members.


My version has 1625 and 9814. I do not understand the difference.



My error.  I had also written a version of the function that stored 
strings instead of ints, and the counts of 3250/19630 was for sets that 
had BOTH.


An exercise for the reader.  Notice that in my brute force algorithm I 
use no global sets.  But I do use an internal list, which is apparently 
unbounded.  So it's feasible to run out of memory.  The challenge is to 
write a similar function that uses no lists, sets, or dicts, just an 
algorithm that detects for an arbitrary sized number whether it's happy 
or not.  (It may not be very quick, but that's yet to be decided.  I'm 
already surprised that the present brute force function  davea1() only 
takes about twice as long as the fancy global caching schemes.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lucky numbers in Python


On 04/30/2015 02:55 PM, Cecil Westerhof wrote:

Because I want the code to work with Python 3 also, the code is now:
 def lucky_numbers(n):
 """
 Lucky numbers from 1 up-to n
 http://en.wikipedia.org/wiki/Lucky_number
 """

 if n < 3:
 return [1]
 sieve = list(range(1, n + 1, 2))
 sieve_index = 1
 while True:
 sieve_len   = len(sieve)
 if (sieve_index + 1) > sieve_len:
 break
 skip_count  = sieve[sieve_index]
 if sieve_len < skip_count:
 break
 del sieve[skip_count - 1 : : skip_count]
 sieve_index += 1
 return sieve

It looks like the list in:
 sieve = list(range(1, n + 1, 2))

does not have much influence in Python 2. So I was thinking of leaving
the code like it is. Or is it better to check and do the list only
with Python 3?



I'd do something like this at top of the module:

try:
range = xrange
except NameError as ex:
pass

then use range as it is defined in Python3.

if that makes you nervous, then define irange = xrange, and if it gets a 
NameError exception, irange = range



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: l = range(int(1E9))


On 04/30/2015 02:48 PM, alister wrote:

On Thu, 30 Apr 2015 20:23:31 +0200, Gisle Vanem wrote:


Cecil Westerhof wrote:


If I execute:
  l = range(int(1E9)

The python process gobbles up all the memory and is killed. The problem
is that after this my swap is completely used, because other processes
have swapped to it. This make those programs more slowly. Is there a
way to circumvent Python claiming all the memory?

By the way: this is CPython 2.7.8.


On what OS? If I try something similar on Win-8.1 and CPython 2.7.5
(32-bit):

   python -c "for i in range(int(1E9)): pass"
Traceback (most recent call last):
  File "", line 1, in 
MemoryError


--gv


also MemoryError on Fedora 21 32 bit



That's presumably because you end up running out of address space before 
you run out of swap space.  On a 64 bit system the reverse will be true, 
unless you have a really *really* large swap file


ulimit is your friend if you've got a program that wants to gobble up 
all of swap space.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is my implementation of happy number OK


On 04/30/2015 11:59 AM, Cecil Westerhof wrote:

I implemented happy_number function:
 _happy_set  = { '1' }
 _unhappy_set= set()

 def happy_number(n):
 """
 Check if a number is a happy number
 https://en.wikipedia.org/wiki/Happy_number
 """

 def create_current(n):
 current_array = sorted([value for value in str(n) if value != '0'])
 return (current_array, ''.join(current_array))

 global _happy_set
 global _unhappy_set

 current_run = set()
 current_array, \
 current_string  = create_current(n)
 if current_string in _happy_set:
 return True
 if current_string in _unhappy_set:
 return False
 while True:
 current_run.add(current_string)
 current_array, \
 current_string = create_current(sum([int(value) ** 2
  for value in 
current_string]))
 if current_string in current_run | _unhappy_set:
 _unhappy_set |= current_run
 return False
 if current_string in _happy_set:
 _happy_set |= current_run
 return True

Besides it need some documentation: is it a good implementation? Or
are there things I should do differently?

To decide for the values from 1 to 1E8 if it is happy or not, takes
280 seconds. Not to bad I think. Also not very good.



First comment, if you're coding a complex implementation like this, take 
the time to do a brute force one as well. Then you can compare the 
results between brute force and your optimized one for all possible 
values, and make sure you haven't introduced any bugs.


My brute force looks like:

#Dave's version, brute force

def davea1(n):
cache = []
anum = str(n)
newnum = 0
while newnum != 1:
newnum = sum(int(i)*int(i) for i in anum)
anum = str(newnum)
if newnum in cache:
return False #not a happy number
cache.append(newnum)
return True  #found a happy number

I then tried an optimized one, and my speed is only about 10% faster 
than yours for 1e7 loops.  I show it anyway, since I think it reads a 
little better.  And readability counts much more than a little performance.


 #optimizations:
#   cached happy and unhappy sets
#   sort the digits, and compare only the sorted values, without zeroes
davea_happy = {1}
davea_unhappy = set()

SQUARES = dict((str(i), i*i) for i in xrange(10))

def davea2(n):
global davea_happy, davea_unhappy
cache = set()
newnum = n
while newnum != 1:
newnum = int("".join(sorted(str(newnum
if newnum in davea_unhappy or newnum in cache:
davea_unhappy |= cache
return False #not a happy number
if newnum in davea_happy:
break
cache.add(newnum)
newnum = sum(SQUARES[ch] for ch in str(newnum))
davea_happy |= cache
return True  #found a happy number

Finally, I did some testing on Jon Ribben's version.  His was 
substantially faster for smaller sets, and about the same for 10*7.  So 
it's likely it'll be slower than yours and mine for 10**8.  But the real 
reason I didn't like it was it produced a much larger set of 
happy_numbers, which could clog memory a lot sooner.  For 10**7 items, I 
had 3250 happy members, and 19630 unhappy.  And Jon had 1418854 happy 
members.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: seek operation in python


On 04/30/2015 04:06 AM, Cecil Westerhof wrote:

Op Thursday 30 Apr 2015 09:33 CEST schreef Chris Angelico:


On Thu, Apr 30, 2015 at 4:27 PM, Cecil Westerhof  wrote:

with open("input.cpp") as f:
lines = f.readlines()
print(lines[7])


Is the following not better:
print(open('input.cpp', 'r').readlines()[7])

Time is the same (about 25 seconds for 100.000 calls), but I find
this more clear.


The significant difference is that the 'with' block guarantees to
close the file promptly. With CPython it probably won't make a lot
of difference, and in a tiny script it won't do much either, but if
you do this on Jython or IronPython or MicroPython or some other
implementation, it may well make a gigantic difference - your loop
might actually fail because the file's still open.


I thought that in this case the file was also closed. But if that is
not the case I should think about this when I switch to another
version as CPython.

I wrote a module where I have:
 def get_indexed_message(message_filename, index):
 """
 Get index message from a file, where 0 gets the first message
 """

 return open(expanduser(message_filename), 
'r').readlines()[index].rstrip()

But this can be used by others also and they could be using Jython or
another implementation. So should I rewrite this and other functions?
Or would it be OK because the open is in a function?



No, it's not going to close the file just because the open is in a 
function.  The "with" construct was designed to help solve exactly this 
problem.  Please use it.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Let exception fire or return None


On 04/30/2015 03:43 AM, Cecil Westerhof wrote:

I have a function to fetch a message from a file:
 def get_indexed_message(message_filename, index):
 """
 Get index message from a file, where 0 gets the first message
 """

 return open(expanduser(message_filename), 
'r').readlines()[index].rstrip()

What is more the Python way: let the exception fire like this code
when index is to big, or catching it and returning None?

I suppose working zero based is OK.



Fire an exception.

One advantage is that the exception will pinpoint exactly which line of 
the function had a problem.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Not possible to hide local variables

2015-04-29 Thread Dave Angel


On 04/29/2015 10:16 AM, Grant Edwards wrote:

On 2015-04-28, Cecil Westerhof  wrote:

If I remember correctly you can not hide variables of a class or make
them read-only?

I want to rewrite my moving average to python. The init is:
 def __init__(self, length):
 if type(length) != int:
 raise ParameterError, 'Parameter has to be an int'
 if n < 0:
 raise ValueError, 'Parameter should be greater or equal 2'
 self.length = length
 self.old_values = []
 self.current_total  = 0

But when someone changes length, old_values, or current_total that
would wreck havoc with my class instance. What is the best way to
handle this?


It's like the punchline to the old doctor joke: if it hurts when you
do that, then don't _do_ that:

   def __init__(self, length):
   if type(length) != int:


Better:  if isinstance(length, int):


   raise ParameterError, 'Parameter has to be an int'
   if n < 0:


Better:   if length < 0:  (since n is undefined)


   raise ValueError, 'Parameter should be greater or equal 2'
   self._length = length
   self._old_values = []
   self._current_total  = 0

The convention is that properties that start with underscores are
private.  They're not hidden, but if people touch them and it breaks
something, it's their fault.  Whether you go all Torvalds on their ass
for doing so is left as an exercise for the reader.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: implicitly concats of adjacent strings does not work with format

2015-04-29 Thread Dave Angel


On 04/29/2015 08:42 AM, Cecil Westerhof wrote:

I have the folowing print statements:
 print(
 'Calculating fibonacci_old, fibonacci_memoize and '
 'fibonacci_memoize once for {0} '.format(large_fibonacci))


 print(
 'Calculating fibonacci_old, fibonacci_memoize and '
 'fibonacci_memoize once for {0} '.format(large_fibonacci) +
 'to determine speed increase')

 print(
 'Calculating fibonacci_old, fibonacci_memoize and '
 'to determine speed increase'
 'fibonacci_memoize once for {0} '.format(large_fibonacci))


 print(
 'Calculating fibonacci_old, fibonacci_memoize and '
 'fibonacci_memoize once for {0} '.format(large_fibonacci)
 'to determine speed increase')

The first three work, but the last gives:
 'to determine speed increase')
 ^
 SyntaxError: invalid syntax

Not very important, because I can use the second one, but I was just
wondering why it goes wrong.



Adjacent string literals are concatenated.  But once you've called a 
method (.format()) on that literal, you now have an expression, not a 
string literal.


You could either change the last line to

 + 'to determine speed increase')

or you could concatenate all the strings before calling the format method:


print(
'Calculating fibonacci_old, fibonacci_memoize and '
'fibonacci_memoize once for {0} '
'to determine speed increase' .format(large_fibonacci))

Something you may not realize is that the addjacent-concatenation occurs 
at compile time, so your third example could be transformed from:


print(
'Calculating fibonacci_old, fibonacci_memoize and '
'to determine speed increase'
'fibonacci_memoize once for {0} '.format(large_fibonacci))

to:
print(
'Calculating fibonacci_old, fibonacci_memoize and '
'to determine speed increase'
'fibonacci_memoize once for {0}'
' '.format(large_fibonacci))


All three literals are combined before format() is called.  Knowing this 
could be vital if you had {} elsewhere in the 9single) literal.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best GUI for Python

2015-04-28 Thread Dave Cook

On 2015-04-26, Cecil Westerhof  wrote:
> I want to use a GUI for Python. When searching I found (beside some
> others) Tkinter and wxPython. From what I found it looks like Tkinter
> is slightly better. What would be the pros/cons of these two? Would
> there be a compelling reason to use another GUI?
>

For cross-platform work, I think it comes down to wx or Qt.  I've used
them on Windows, Mac and Linux.  Qt has the more polished and
consistent API, and is more popular with the big scientific Python
distros. wx has the most liberal license, and I think having to deal
with only one Python binding is somewhat an advantage.  I've gone back
and forth between them, and could probably live with either one.

Gtk is also worth looking at if you only care about Linux.

Dave Cook
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best GUI for Python

2015-04-28 Thread Dave Farrance

Christian Gollwitzer  wrote:

>Yes, the default theme is terrible on Linux (Mac & Windows uses native 
>widgets). There are additional themes available, which are buried in 
>some packages and a bit difficult to install, but give reasonable 
>approximations to the QT look; I'm talking about plastik, for instance 
>http://wiki.tcl.tk/24094 (1st picture - the layout of the test is 
>terrible, but the widgets do look good). For some reason I've got no 
>insights, these themes are very slow (they are based on displaying 
>pre-rendered PNG images for the widget bits and pieces).
>Yet another possibility are the tileQT and tileGTK packages, which 
>render the widgets using QT and GTK, but of course this introduces these 
>dependencies and it might be simpler to just use QT or GTK natively.

Yes, in the case of a more complex control-panel interface, it becomes
worthwhile to design it with a GUI designer like Glade anyway, so that
would give it the "gi" GTk3 widget set which looks fine.

Tkinter is OK if you want to make something utilitarian like an
oscilloscope interface and it does seem to be the easiest way to build
an interface without a GUI builder.

Kivy isn't bad, being fairly easy to use without a GUI designer, and
having a modern flat smartphone-style widget set, if you like that sort
of thing. You're expected to give the buttons logos and different
colours.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Matplotlib] Ploting an exponential distribution frequency curve

2015-04-26 Thread Dave Farrance

Mario Figueiredo  wrote:

>Other than replacing the random module with the probability density
>function for the exponential distribution, do you have a suggestion of
>how I could smooth the curve?

Moving average. Try:

def movingaverage(interval, window_size):
window= numpy.ones(int(window_size))/float(window_size)
return numpy.convolve(interval, window, 'same')

y_av = movingaverage(y,10)

Note that you'd get problems at the start and end of the curve if it's
non-zero there, which is difficult to avoid with noise-reduction.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Panda data read_csv returns 'TextFileReader' object

2015-04-24 Thread Dave Angel


On 04/24/2015 04:04 PM, Kurt wrote:

Isn't the call pd.read_csv(filepath,...) suppose to return a dataframe, not 
this other object? I keep getting the following error when I try to view the 
attribute head.

AttributeError: 'TextFileReader' object has no attribute 'head'

I reference pandas as pd and df is suppose to define itself. I had this working 
at one time.



Please supply the Python version, the pandas version, and the code you 
ran.  It can also be useful to show how/where you installed pandas, like 
the URL you used, or the apt-get, or the pip command, or whatever.


Then show the complete error traceback, not just a summary.  And if the 
error really is on the line you partially supplied above, what's the 
type and contents of filepath?  What are the other arguments' values?


The function I suspect you're trying to call is:

pandas.io.parsers.read_csv(filepath, ...)

but I can't tell from your description.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: can you help guys?

2015-04-24 Thread Dave Angel


On 04/24/2015 04:29 AM, brokolists wrote:

24 Nisan 2015 Cuma 02:20:12 UTC+3 tarihinde Steven D'Aprano yazdı:

On Fri, 24 Apr 2015 01:51 am, brokolists wrote:


my problem is i m working with very long float numbers and i use
numberx =float(( input( 'enter the number\n ')))
after i use this command when i enter something more than 10 or 11 digits
it uses like 1e+10 or something like that but i have to calculate it
without scientific 'e' type. what can i do to solve this? (sorry for my
bad english)







i fixed the "e" problem by using digits

getcontext().prec=20
it works fine but in my algoritm it was checking the answers by their lenght so 
this created a new problem by giving all the answers with 20 digits. I might 
fix this but i have searched for 4 hours just how to get rid of 'e' . i think 
there must be a easier way to do this.



We could be a lot more helpful if you just spelled it out.  digits and 
getcontext() are not builtins, so we don't really know how you're using 
them, except by guessing.


You've changed the code, probably by using one of the many suggestions 
here.  But what's it look like now?


You should have started the thread by telling us the Python version. 
I'm guessing you're using python 3.x



In Python 3.x, input() returns a string.  So you can tell how much the 
user typed by doing a len() on that string.  Once you convert it into 
some other form, you're choosing the number of digits partly by how you 
convert it.


Can you state what your real assignment is?  And what code you have so 
far?  And just what is wrong with it, that you need to change?


For example, if the "correct" answer for your user is "7.3"  and the 
user types  "7.300" is that supposed to be right, or wrong?





--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: A question on the creation of list of lists

2015-04-23 Thread Dave Angel


On 04/23/2015 08:36 AM, Gregory Ewing wrote:

Jean-Michel Pichavant wrote:

From: "subhabrata banerji" 

list_of_files = glob.glob('C:\Python27\*.*')

 >

1/ Your file pattern search will not get files that do not have any
dot in
their name


Actually, on Windows, it will. (This is for compatibility with
MS-DOS 8.3 filenames, where the dot wasn't actually stored, so
there was no distinction between a filename with a dot and an
empty extension, and a filename with no dot.)



That's certainly true at the command line in Windows DOS box, but I'd be 
astounded if glob used the same logic, even on Winodows.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: May I drop list bracket from list?

2015-04-23 Thread Dave Angel


On 04/23/2015 06:11 AM, subhabrata.bane...@gmail.com wrote:

Dear Group,

I am trying to read a list of files as
list_of_files = glob.glob('C:\Python27\*.*')
Now I am trying to read each one of them,
convert into list of words, and append to a list
as.

list1=[]
for file in list_of_files:
   print file
   fread1=open(file,"r").read()
   fword=fread1.split()
   list1.append(fword)

Here the list is a list of lists, but I want only one list not
list of lists.

I was thinking of stripping it as, str(list1).strip('[]')

but in that case it would be converted to string.

Is there a way to do it. I am using Python27 on Windows7 Professional.
Apology for an indentation error.

If anybody may please suggest.



You're first problem is the name of your variable.  fword implies it's a 
string, but it's really a list.  So when you do:

 list1.append(fword)

you're appending a list to a list, which gives you nested lists.  Sounds 
like you want

 list1.extend(fword)



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Diff between object graphs?

2015-04-22 Thread Dave Angel


On 04/22/2015 09:46 PM, Chris Angelico wrote:

On Thu, Apr 23, 2015 at 11:37 AM, Dave Angel  wrote:

On 04/22/2015 09:30 PM, Cem Karan wrote:



On Apr 22, 2015, at 8:53 AM, Peter Otten <__pete...@web.de> wrote:


Another slightly more involved idea:

Make the events pickleable, and save the simulator only for every 100th
(for
example) event. To restore the 7531th state load pickle 7500 and apply
events 7501 to 7531.



I was hoping to avoid doing this as I lose information.  BUT, its likely
that this will be the best approach regardless of what other methods I use;
there is just too much data.



Why would that lose any information???


It loses information if event processing isn't perfectly deterministic.


Quite right.  But I hadn't seen anything in this thread to imply that.

I used an approach like that on the Game of Life, in 1976.  I saved 
every 10th or so state, and was able to run the simulation backwards by 
going forward from the previous saved state.  In this case, the analogue 
of the "event" is determined from the previous state.  But it's quite 
similar, and quite deterministic.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Diff between object graphs?

2015-04-22 Thread Dave Angel


On 04/22/2015 09:30 PM, Cem Karan wrote:


On Apr 22, 2015, at 8:53 AM, Peter Otten <__pete...@web.de> wrote:


Another slightly more involved idea:

Make the events pickleable, and save the simulator only for every 100th (for
example) event. To restore the 7531th state load pickle 7500 and apply
events 7501 to 7531.


I was hoping to avoid doing this as I lose information.  BUT, its likely that 
this will be the best approach regardless of what other methods I use; there is 
just too much data.



Why would that lose any information???


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python as shell

2015-04-22 Thread Dave Farrance

Cecil Westerhof  wrote:

>I am working again with Python and I am impressed again. ;-)
>
>I thought there was a Python shell that could be used instead of Bash
>(or whichever shell you are using), but I can not find anything about
>it. Am I wrong, or are my search engine skills so bad?

You're probably thinking of IPython which has some shell-like commands,
like ls and cd, and you can execute system commands by prefixing them
with "!".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: multiprocessing module and matplotlib.pyplot/PdfPages

2015-04-21 Thread Dave Angel


On 04/21/2015 07:54 PM, Dennis Lee Bieber wrote:

On Tue, 21 Apr 2015 18:12:53 +0100, Paulo da Silva
 declaimed the following:




Yes. fork will do that. I have just looked at it and it is the same as
unix fork (module os). I am thinking of launching several forks that
will produce .png images and at the end I'll call "convert" program to
place those .png files into a pdf book. A poor solution but much faster.



To the best of my knowledge, on a UNIX-type OS, multiprocessing /uses/
fork() already. Windows does not have the equivalent of fork(), so
multiprocessing uses a different method to create the process
(conceptually, it runs a program that does an import of the module followed
by a call to the named method -- which is why one must follow the

if __name__  ...

bit prevent the subprocess import from repeating the original main program.



The page:

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

indicates that there are 3 ways in which a new process may be started. 
On Unix you may use any of the three, while on Windows, you're stuck 
with spawn.


I *think* that in Unix, it always does a fork.  But if you specify 
"spawn" in Unix, you get all the extra overhead to wind up with what 
you're describing above.  If you know your code will run only on Unix, 
you presumably can get much more efficiency by using the fork 
start-method explicitly.


I haven't done it, but it would seem likely to me that forked code can 
continue to use existing global variables.  Changes to those variables 
would not be shared across the two forked processes.  But if this is 
true, it would seem to be much easier way to launch the second process, 
if it's going to be nearly identical to the first.


Maybe this is just describing the os.fork() function.
   https://docs.python.org/3.4/library/os.html#os.fork

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: multiprocessing module and matplotlib.pyplot/PdfPages

2015-04-21 Thread Dave Angel


On 04/20/2015 10:14 PM, Paulo da Silva wrote:

I have program that generates about 100 relatively complex graphics and
writes then to a pdf book.
It takes a while!
Is there any possibility of using multiprocessing to build the graphics
and then use several calls to savefig(), i.e. some kind of graphic's
objects?



To know if this is practical, we have to guess about the code you 
describe, and about the machine you're using.


First, if you don't have multiple cores on  your machine, then it's 
probably not going to be any faster, and might be substantially slower. 
 Ditto if the code is so large that multiple copies of it will cause 
swapping.


But if you have 4 cores, and a processor=bound algorithm, it can indeed 
save time to run 3 or 4 processes in parallel.  You'll have to write 
code to parcel out the parts that can be done in parallel, and make a 
queue that each process can grab its next assignment from.


There are other gotchas, such as common code that has to be run before 
any of the subprocesses.  If you discover that each of these 100 pieces 
has to access data from earlier pieces, then you could get bogged down 
in communications and coordination.


If the 100 plots are really quite independent, you could also consider 
recruiting time from multiple machines.  As long as the data that needs 
to go between them is not too large, it can pay off big time.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Opening Multiple files at one time

2015-04-21 Thread Dave Angel


On 04/21/2015 03:56 AM, subhabrata.bane...@gmail.com wrote:



Yes. They do not. They are opening one by one.
I have some big chunk of data I am getting by crawling etc.
now as I run the code it is fetching data.
I am trying to fetch the data from various sites.
The contents of the file are getting getting stored in
separate files.
For example, if I open the site of "http://www.theguardian.com/international";, then the result may be stored 
in file in file "file1.txt", and the contents of site "http://edition.cnn.com/";, may be stored in 
file "file2.txt".

But the contents of each site changes everyday. So everyday as you open these 
sites and store the results, you should store in different text files. These 
text files I am looking to be created on its own, as you do not know its 
numbers, how many numbers you need to fetch the data.

I am trying to do some results with import datetime as datetime.datetime.now()
may change everytime. I am trying to put it as name of file. But you may 
suggest better.



To get the text version of today's date, use something like:

import datetime
import itertools
SUFFIX = datetime.datetime.now().strftime("%Y$%m%d")

To write a filename generator that generates names sequentially  (untested):

def filenames(suffix=SUFFIX):
for integer in itertools.count(1):
yield "{0:04d}".format(integer) + "-" + SUFFIX


for filename in filenames():
f = open (filename, "w")
...Do some work here which writes to the file, and does "break"
... if we don't need any more files
f.close()

Note that this is literally open-ended.  If you don't put some logic in 
that loop which will break, it'll write files till your OS stops you, 
either because of disk full, or directory too large, or whatever.


I suggest you test the loop out by using a print() before using it to 
actually create the files.


In the format above, I used 4 digits, on the assumption that usually 
that is enough.  If you need more than that on the occasional day, it 
won't break, but the names won't be nicely sorted when you view them.



If this were my problem, I'd also use a generator for the web page 
names.  If you write that generator, then you could do something like 
(untested):


for filename, url in zip(filenames(), urlames():
f = open (filename, "w")
... process the url, writing to file f
f.close()

In this loop, it'll automatically end when you run out of urlnames.


I also think you should consider making the date the directory name you 
use, rather than putting many days files in a single directory.  But 
this mainly affects the way you concatenate the parts together.  You'd 
use os.path.join() rather than  "+" to combine parts.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Opening Multiple files at one time

2015-04-20 Thread Dave Angel


On 04/20/2015 07:59 AM, subhabrata.bane...@gmail.com wrote:

Dear Group,

I am trying to open multiple files at one time.
I am trying to do it as,

  for item in  [ "one", "two", "three" ]:
f = open (item + "world.txt", "w")
f.close()

This is fine.


But it does not open multiple files at one time.  Perhaps you'd better 
explain better what you mean by "at one time."



But I was looking if I do not know the number of
text files I would create beforehand,


So instead of using a literal list [ "one", "two... ]
you construct one, or read it in from disk, or use sys.argv.  What's the 
difficulty?  Nothing in the code fragment you show cares how long the 
list is.



 so not trying xrange option

also.


No idea what "the xrange option" means.



And if in every run of the code if the name of the text files have
to created on its own.


So figure out what you're going to use for those names, and write the 
code to generate them.




Is there a solution for this?


For what?


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Problem running Python 2.7 on Ubuntu 10.04

2015-04-20 Thread Dave Farrance

David Aldrich  wrote:

>Hi
>
>I wonder if someone could help me with this problem please?
>
>On an Ubuntu 10.04 platform, I want to run the latest version of Meld, which 
>is a Python program.
>
>Ubuntu 10.04 runs Python 2.6 as standard.  Meld requires Python 2.7.  So I 
>have installed Python 2.7 under /usr/local/bin and Python 2.7 runs ok there.
>
>But when I try to run Meld I see:
>
>-> /usr/local/bin/python2.7 ~/meld/bin/meld
>Cannot import: GTK+
>No module named gi

My understanding is that "gi" is GTk3 and I suspect that you're not
going to successfully install that on Ubuntu 10 which is a GTk2 distro.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python and lotus notes

2015-04-20 Thread Dave Angel


On 04/20/2015 04:29 AM, gianluca.pu...@gmail.com wrote:

Hi,



Hi and welcome.
I don't know Lotus Notes, but i can at least comment on some of your 
code, pointing out at least some problems.



i am having a problem when i try to access lotus notes with python, think i do 
all ok but it seems something is going wrong because i can't print any db title 
even if i've opened the .nsf file.

My code:

import win32com.client
from win32com.client import Dispatch

notesServer='Mail\wed\new\IT'


You're using backslashes in literal strings when there's nothing escaped 
there.  So you need to either double them or use a raw string.  It's 
also possible that you could use forward slashes, but I can't be sure.


I'd simply use
notesServer = R'Mail\wed\new\IT'



notesFile= 'Apps\Mydb.nsf'
notesPass= 'mypass'

session = Dispatch('Lotus.NotesSession')
session.Initialize(notesPass)
print ""

db = session.getDatabase(notesServer, notesFile)

print db.Open


You forgot the parentheses.  So you're not calling the Open function, 
you're just printing it.



print db.IsOpen
print db.Title

but i receve:

>>
False

Please can u help me?
GL



There may be other things, but if you fix the three lines, it'll 
probably get further.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: do you guys help newbies??


On 04/19/2015 09:37 PM, josiah.l...@stu.nebo.edu wrote:

On Wednesday, November 27, 2002 at 4:01:02 AM UTC-7, John Hunter wrote:

"malik" == malik martin  writes:


 malik>i'm having a simple problem i guess.  but i still dont
 malik> know the answer. anyone see anything wrong with this code?
 malik> i think it's the line in the while loop that starts with
 malik> "averagea = ..."  because it will execute once but once the
 malik> loop goes around one more time it gets an error. is there
 malik> it because averagea has been assigned a definate value? if
 malik> so how do i record the value but reset averagea back to 0(
 malik> i guess it would be 0)

 malik>any help would be apreciated. thanks!!

I think you forgot to post your code.   Makes it harder.

JDH




And I think you forgot to check the date of the post to which you're 
responding.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python and fortran Interface suggestion


On 04/19/2015 11:56 AM, pauld11718 wrote:

I shall provide with further details

Its about Mathematical modelling of a system. Fortran does one part and python 
does the other part (which I am suppose to provide).
For a time interval tn --> t_n+1, fortran code generates some values, for which my 
python code accepts it as an input. It integrates some data for the same time step tn 
--> tn+1 and fortran computes based on this for the next time step t_n+1 --> 
t_n+2..and this is how loop continues.

Its the fortran code calling my Python executable at all the time interval.


Then you'd better find out how it's calling your executable.  Calling it 
is very different than starting it.  The whole import x, y,z is done 
only upon initial startup of the python code.  You can then call the 
Python code as many times as you like without incurring that overhead again.


Naturally, if the guy who designed the Fortran code didn't think the 
same way, and is unavailable for further tweaking, you've got a problem.




So,
1. Is it about rebuilding every thing using cx_freeze kind of stuff in windows?


Don't worry about how you get installed until after you figure out how 
you're going to get those calls and data back and forth between the 
Fortran code and your own.




2. As interfacing between fortran/python is via .dat file, so on every 
time-step python executable is called it shall open()/close() .dat file. This 
along with all those
from  import *'s
are unnecessary.


Have you got specs on the whole dat file thing?  How will you know it's 
time to examine the file again?  As long as you get notified through 
some other mechanism, there's no problem in both programs having an open 
handle on the shared file.


For that matter, the notification can be implicit in the file as well.

But is there a designer behind this, or is the program intending to deal 
with something else and you're just trying to sneak in the back door?





3. I dont have access to fortran code. I shall provide the python executable 
only, which will take input from .dat file and write to another .dat file. So, 
what is your suggestion regarding socket/pipe and python installation on RAM? I 
have no idea with those.



Not much point in opening a pipe if the other end isn't going to be 
opened by the Fortran code.



Isn't there a better way to handle such issues?



Sure, there are lots of ways.  But if the Fortran code is a closed book, 
you'll have to find out how it was designed, and do all the adapting at 
your end.


If it becomes open again, then you have the choice between having one of 
your languages call functions in the other (ie. single process), having 
some mechanism (like queues, or sockets) between two separately started 
processes, and having one program launch the other repeatedly.  That 
last is probably the least performant choice.


If you google "python Fortran" you'll get lots of hits.  You could start 
reading to see what your choices are.  But if the Fortran designer has 
left the room, you'll be stuck with whatever he put together.


And chances are very high that you'll need to develop, and certainly 
test, some parts of the project on the Windows machine, and particularly 
with the Windows C/Fortran compilers.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/19/2015 09:02 AM, Steven D'Aprano wrote:

On Sun, 19 Apr 2015 04:08 am, Albert van der Horst wrote:


Fire up a lowlevel interpreter like Forth. (e.g. gforth)


Yay! I'm not the only one who uses or likes Forth!

Have you tried Factor? I'm wondering if it is worth looking at, as a more
modern and less low-level version of Forth.



I also like Forth (since 83), but haven't done much in the last decade.

I was newsletter editor for our local FIG for many years.

I have met and debated with Elizabeth Rather, and been a "third hand" 
for Chuck Moore when he was re-soldering wires on his prototype Forth board.


You can see my name in the X3J14 standard:
   https://www.taygeta.com/forth/dpans1.htm#x3j14.membership


I'd be interested in a "more modern" Forth, but naturally, as a member 
of band of rugged individualists, I wonder if it can possibly satisfy 
more than one of us.



http://factorcode.org/

That site is my first time I recall seeing "concatenative" as a type of 
language.  Interesting way of thinking of it.  I just call it RPN, and 
relate it to the original HP35 calculator ($400, in about 1972).


From the overview, it looks like they're at least aiming at what I 
envisioned as the next Forth I wanted to use.  Instead of putting ints 
and addresses on the stack, you put refs to objects, in the Python 
sense.  Those objects are also gc'ed.  I don't know yet whether 
everything is an object, or whether (like Java), you have boxed and 
unboxed thingies.


Sounds interesting, and well worth investigating.  thanks for pointing 
it out.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: New to Python - block grouping (spaces)


On 04/19/2015 07:38 AM, BartC wrote:


Perhaps you don't understand what I'm getting at.

Suppose there were just two syntaxes: C-like and Python-like (we'll put
aside for a minute the question of what format is used to store Python
source code).

Why shouldn't A configure his editor to display a Python program in
C-like syntax, and B configure their editor to use Python-like tabbed
syntax?

A can write code in the preferred syntax, and B can view/modify exactly
the same code in /their/ preferred syntax. What would be the problem?
(The actual stored representation of the program would be in one of
those two styles, or something else entirely; Lisp-like syntax for
example. It doesn't matter because no-one would care.

(I think much of the problem that most languages are intimately
associated with their specific syntax, so that people can't see past it
to what the code is actually saying. a=b, a:=b, b=>a, (setf a b),
whatever the syntax is, who cares? We just want to do an assignment!)



If you make enough simplifying assumptions, of course it's easy and natural.

Assume that a text editor is the only way you'll be viewing the source 
code.  You and your coworkers are willing to each use a prescribed text 
editor, rather than your previous favorite one that doesn't happen to 
support the customization you're suggesting here.  You're not going to 
use a version control system, nor diff programs.  You're not going to 
post your code in messages on the internet.


You're not going to display error messages showing source code, from a 
running system.  You're not going to use the interactive debugger.


You're not going to copy code from one place to another without copying 
sufficient context to be able to reconstruct the funny syntax.


You're going to permit only one such variation, and it's just big enough 
that it's always obvious which version of the code is being examined (by 
programs or by humans) [1]


You're not going to use eval()  [good].  You're not going to examine 
source code that comes from 3rd party or the standard library.


The changes you want are all completely reversible, regardless of 
interspersed comments, and when reversed, preserve spacing and 
characters completely in the way that each user expects to see the code. 
 And they are reversible even if you only have a small fragment of the 
code.


I implemented something analogous to such a system 40 years ago.  But on 
that system, nearly all of the restrictions above applied.  There was no 
clipboard, no version control, no internet.  Programs had to fit in 64k. 
Each line stood completely on its own, so context was not a problem.  i 
wrote the text editor, much of the interactive debugger, the listing 
utility, the pretty printer, the cross reference program, etc.  So they 
were consistent.  Further, we had a captive audience -- if they didn't 
like it, they could buy the competitor's product, which had similar 
constraints.  Would I do things differently now?  You bet I would.  At 
least if I could upgrade the memory addressability to a few megs.



For the purposes you've described so far, I'd suggest just writing two 
translators, one a mirror image of the other.  Invoke them automatically 
on load and save in your personal editor  And learn to live with both 
syntaxes, because it will leak.  And if it's really not reversible, 
don't use them when you're editing somebody else's code.  And even if it 
is reversible, don't use them when you edit code you plan to post on the 
internet.


[1] So for example  a=b can not mean assignment in one version, and a 
comparison in the other.  Because then you'd need to either discover 
that it's a line by itself, or within an if expression, or ...  But you 
could arrange two disjoint sets of symbols readily enough.  In your 
version, require either := or ==, and make = just plain illegal.


PS.  I haven't examined the costs in tool development to support this. 
Just whether it's practical.  You can easily discount any one of these 
constraints, but taken as a whole, they very much limit what you can do.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: HELP! How to return the returned value from a threaded function

2015-04-18 Thread Dave Angel


On 04/18/2015 01:07 PM, D. Xenakis wrote:

Maybe this is pretty simple but seems I am stuck...

def message_function():
 return "HelloWorld!"


def thread_maker():
 """
 call message_function()
 using a new thread
 and return it's "HelloWorld!"
 """
 pass


Could someone please complete above script so that:

thread_maker() == "HelloWorld!"

Please import the library you suggest too.
I tried threading but I could not make it work.

THX!



The first question is why you are even starting extra threads.  They are 
only useful if they do concurrent work.  So if you're starting a thread, 
then not doing anything yourself till the thread terminates, you've 
gained absolutely nothing.  And greatly complicated your code besides.


In fact, even if you start multiple threads, and/or continue doing your 
own work in the main thread, you might not gain anything in CPython, 
because of the GIL.


But if I assume you've justified the use of threads, or your 
boss/instructor has mandated them, then you still have to decide what 
the flow of logic is going to mean.


If you're going to start a bunch of threads, wait a while, and do 
something with the "return values" of some subset of them, then all you 
need is a way to tell whether those threads have yet posted their 
results.  If there's exactly one result per thread, then all you need is 
a global structure with room for all the results, and with an initial 
value that's recognizably different.


So fill a list with None elements, and tell each thread what element of 
the list to update.  Then in your main thread, you trust any list 
element that's no longer equal to None.


Alternatively, you might decide your main program will wait till all the 
threads have terminated.  In that case, instead of checking for None, 
simply do a join on all the threads before using the results.


But many real threaded programs use much more complex interactions, and 
they reuse threads rather than expecting them to terminate as soon as 
one result is available.  So there might not be a single "result" per 
thread, and things get much more complex.  This is where you would 
normally start studying queues.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Failed to import a "pyd: File When python intepreter embed in C++ project

2015-04-17 Thread Dave Angel


On 04/17/2015 01:17 PM, saadaouijihed1...@gmail.com wrote:

I have a swig module (.pyd).I followed the steps but it doesn't work
please help me.



First, unplug the computer and remove the battery.  Then if it's still 
burning, douse it with a fire extinguisher.


If your symptoms are different, you'll have to spell it out.  Like 
Python version, OS, complete stack trace.  And anything else that may be 
needed to diagnose the problem.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Converting text file to different encoding.

2015-04-17 Thread Dave Angel


On 04/17/2015 10:48 AM, Dave Angel wrote:

On 04/17/2015 09:19 AM, subhabrata.bane...@gmail.com wrote:



>>> target = open("target", "w")


It's not usually a good idea to use the same variable for both the file
name and the opened file object.  What if you need later to print the
name, as in an error message?


Oops, my error.  Somehow my brain didn't notice the quote marks, until I 
reread my own message online.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Converting text file to different encoding.

2015-04-17 Thread Dave Angel

On 04/17/2015 09:19 AM, subhabrata.bane...@gmail.com wrote:

I am having few files in default encoding. I wanted to change their encodings,
preferably in "UTF-8", or may be from one encoding to any other encoding.

You neglected to specify what Python version this is for.  Other 
information that'd be useful is whether the file size is small enough 
that two copies of it will all fit reasonably into memory.

I'll assume it's version 2.7, because of various clues in your sample 
code.  But if it's version 3.x, it could be substantially easier.

I was trying it as follows,

>>> import codecs
>>> sourceEncoding = "iso-8859-1"
>>> targetEncoding = "utf-8"
>>> source = open("source1","w")

mode "w" will truncate the source1 file, leaving you nothing to process. 
 i'd suggest "r"

>>> target = open("target", "w")

It's not usually a good idea to use the same variable for both the file 
name and the opened file object.  What if you need later to print the 
name, as in an error message?

>>> target.write(unicode(source, sourceEncoding).encode(targetEncoding))

I'd not recommend trying to do so much in one line, at least until you 
understand all the pieces.  Programming is not (usually) a contest to 
write the most obscure code, but rather to make a program you can still 
read and understand six months from now.  And, oh yeah, something that 
will run and accomplish something.

>
> but it was giving me error as follows,
> Traceback (most recent call last):
>File "", line 1, in 
>  target.write(unicode(source, sourceEncoding).encode(targetEncoding))
> TypeError: coercing to Unicode: need string or buffer, file found

if you factor this you will discover your error.  Nowhere do you read 
the source file into a byte string.  And that's what is needed for the 
unicode constructor.  Factored, you might have something like:

 encodedtext = source.read()
 text = unicode(source, sourceEncoding)
 reencodedtext = text.encode(targetEncoding)
 target.write(encodedText)

Next, you need to close the files.

source.close()
target.close()

There are a number of ways to improve that code, but this is a start.

Improvements:

 Use codecs.open() to open the files, so encoding is handled 
implicitly in the file objects.

 Use with... syntax so that the file closes are implicit

 read and write the files in a loop, a line at a time, so that you 
needn't have all the data in memory (at least twice) at one time.  This 
will also help enormously if you encounter any errors, and want to 
report the location and problem to the user.  It might even turn out to 
be faster.

 You should write non-trivial code in a text file, and run it from 
there.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: ctypes: using .value or .value() doesn't work for c_long

2015-04-15 Thread Dave Angel


On 04/15/2015 03:48 PM, IronManMark20 wrote:

I am using ctypes to call a few windll funcions. One of them returns a c_long 
object. I want to know what number the function returns.

Problem is, when I try foo.value , it gives me this:

AttributeError: LP_c_long object has no attribute value.

Any idea of what could cause this?



I don't use Windows, so I can't try it, especially since you didn't 
include any code .  And apparently whatever function you're using 
returns a pointer to a c_long, not a c_long.


I see Ian has given you an answer.  But let me show you how you might 
have come up with it on your own, for next time.


The fragment of the error you include says that the instance of the 
class LP_c_long doesn't have an attribute called "value".  But it 
undoubtedly has other attributes, so you could look with dir


print(dir(foo))

Presumably from there you'll see that you need the 'contents' attribute. 
 Then you print that out, and/or dir a dir() on it, and you see that 
you now have a value attribute.



https://docs.python.org/3/library/ctypes.html

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there functions like the luaL_loadfile and luaL_loadbuffer in lua source to dump the lua scripts in Python's source?

2015-04-14 Thread Dave Angel


On 04/14/2015 08:07 AM, zhihao chen wrote:

HI,I  want to dump the python script when some application(android's app)
use the python engine.they embedding python into its app.


  I can dump the script from an app which use the lua engine through the
luaL_loadbuffer or luaL_loadfile (just hook this function,and dump the lua
script by the (char *)buffer and size_t )

And,I want to know that:

Is there functions like the luaL_loadfile or luaL_loadbuffer on python's
source to read the python's file,so I cann't through this function to get
the (char *)buffer and size so to get the *.py or *.pyc ?

In other words,which C/C++ function contribute to load the all python's
script or pyc，I want to dump the scripts in that.



I know nothing about lua, but the __file__ attribute on most user 
modules will tell you the filename it's loaded from.  Naturally, that 
assumes it actually came from a file, and a few other things.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pickle based workflow - looking for advice

2015-04-13 Thread Dave Angel


On 04/13/2015 10:58 AM, Fabien wrote:

Folks,



A comment.  Pickle is a method of creating persistent data, most 
commonly used to preserve data between runs.  A database is another 
method.  Although either one can also be used with multiprocessing, you 
seem to be worrying more about the mechanism, and not enough about the 
problem.



I am writing a quite extensive piece of scientific software. Its
workflow is quite easy to explain. The tool realizes series of
operations on watersheds (such as mapping data on it, geostatistics and
more). There are thousands of independent watersheds of different size,
and the size determines the computing time spent on each of them.


First question:  what is the name or "identity" of a watershed? 
Apparently it's named by a directory.  But you mention ID as well.  You 
write a function A() that takes only a directory name. Is that the name 
of the watershed?  One per directory?  And you can derive the ID from 
the directory name?


Second question, is there any communication between watersheds, or are 
they totally independent?


Third:  this "external data", is it dynamic, do you have to fetch it in 
a particular order, is it separated by watershed id, or what?


Fourth:  when the program starts, are the directories all empty, so the 
presence of a pickle file tells you that A() has run?  Or is there some 
other meaning for those files?




Say I have the operations A, B, C and D. B and C are completely
independent but they need A to be run first, D needs B and C, and so
forth. Eventually the whole operations A, B, C and D will run once for
all,


For all what?


but of course the whole development is an iterative process and I
rerun all operations many times.


Based on what?  Is the external data changing, and you have to rerun 
functions to update what you've already stored about them?  Or do you 
just mean you call the A() function on every possible watershed?




(I suddenly have to go out, so I can't comment on the rest, except that 
choosing to pickle, or to marshall, or to database, or to 
custom-serialize seems a bit premature.  You may have it all clear in 
your head, but I can't see what the interplay between all these calls to 
one-letter-named functions is intended to be.)



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Excluding a few pawns from the game

2015-04-13 Thread Dave Angel


On 04/13/2015 07:30 AM, userque...@gmail.com wrote:

I am writing a function in python, where the function excludes a list of pawns 
from the game. The condition for excluding the pawns is whether the pawn is 
listed in the database DBPawnBoardChart. Here is my code:

   def _bring_bigchart_pawns(self, removed_list=set(), playing_amount=0):
 chart_pawns = DBPawnBoardChart.query().fetch()
 chart_pawns_filtered = []
 for bigchart_pawn in chart_pawns:
 pawn_number = bigchart_pawn.key.number()
 db = DBPawn.bring_by_number(pawn_number)
 if db is None:
 chart_pawn.key.delete()
 logging.error('DBPawnBoardChart entry is none for chart_pawn = 
%s' % pawn_number)
 if pawn_number in chart_pawns:
 chart_pawn.add(pawn_number)
 else:
 exclude_pawn_numbers = ['1,2,3']
 chart_pawns_filtered.append(chart_pawn)
 pawn_numbers = [x.key.number() for x in chart_pawns_filtered]
 return pawn_numbers, chart_pawns, exclude_pawn_numbers

If the pawn is listed in DBPawnBoardChart it should be added to the game or 
else it should be excluded. I am unable to exclude the pawns,the 
DBPawnBoardChart contains Boardnumber and Pawnnumber. What necessary changes 
should I make ?



Looks to me like an indentation error.  About 9 of those lines probably 
need to be included in the for loop, so they have to be indented further in.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: installing error in python


On 04/13/2015 01:38 AM, Mahima Goyal wrote:

error of corrupted file or directory is coming if i am installing
python for 64 bit.



And what OS is that on, and how long has it been since you've done a 
file system check on the drive?



For that matter, what version of Python are you installing, and where 
are you getting the download for it?


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: find all multiplicands and multipliers for a number


On 04/13/2015 01:25 AM, Paul Rubin wrote:

Dave Angel  writes:

But doesn't math.pow return a float?...
Or were you saying bignums bigger than a float can represent at all?  Like:

x = 2**1 -1  ...
math.log2(x)

1.0


Yes, exactly that.


Well that value x has some 3300 digits, and I seem to recall that float 
only can handle 10**320 or so.  But if the point of all this is to 
decide when to stop dividing, I think our numbers here are somewhere 
beyond the heat death of the universe.


  Thus (not completely tested):


 def isqrt(x):
 def log2(x): return math.log(x,2)  # python 2 compatibility
 if x < 1e9:


Now 10**9 is way below either limit of floating point.  So i still don't 
know which way you were figuring it.  Just off the top of my head, I 
think 10**18 is approx when integers don't get exact representation, and 
10**320 is where you can't represent numbers as floats at all.



return int(math.ceil(math.sqrt(x)))
a,b = divmod(log2(x), 1.0)
c = int(a/2) - 10
d = (b/2 + a/2 - c + 0.001)
# now c+d = log2(x)+0.001, c is an integer, and
 # d is a float between 10 and 11
s = 2**c * int(math.ceil(2**d))
return s

should return slightly above the integer square root of x.  This is just
off the top of my head and maybe it can be tweaked a bit.  Or maybe it's
stupid and there's an obvious better way to do it that I'm missing.



If you're willing to use the 10**320 or whatever it is for the limit, I 
don't see what's wrong with just doing floating point sqrt.  Who cares 
if it can be an exact int, since we're just using it to get an upper limit.


And I can't figure out your code at this hour of night, but it's much 
more complicated than Newton's method would be anyway.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: find all multiplicands and multipliers for a number

On 04/12/2015 11:30 PM, Paul Rubin wrote:

Dave Angel  writes:

If I were trying to get a bound for stopping the divide operation, on
a value too large to do exact real representation, I'd try doing just
a few iterations of Newton's method.

Python ninja trick: math.log works on bignums too large to be
represented as floats ;-)

But doesn't math.pow return a float?   Af first crack I figured it was 
because I had supplied math.e as the first argument.  But I have the 
same problem with  (python 3.4)

x = 2596148429267413814265248164610047
print( math.pow(2, math.log2(x)) )
2.596148429267414e+33

Or were you saying bignums bigger than a float can represent at all?  Like:

>>> x = 2**1 -1
>>> len(str(x))
3345
>>> math.log2(x)
1.0
>>> math.pow(2, math.log2(x)//2)
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: math range error

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: find all multiplicands and multipliers for a number


On 04/12/2015 09:56 PM, Paul Rubin wrote:

Marko Rauhamaa  writes:

And in fact, the sqrt optimization now makes the original version 20%
faster: ...
 bound = int(math.sqrt(n))


That could conceivably fail because of floating point roundoff or
overflow, e.g. fac(3**1000).  A fancier approach to finding the integer
square root might be worthwhile though.



If I were trying to get a bound for stopping the divide operation, on a 
value too large to do exact real representation, I'd try doing just a 
few iterations of Newton's method.  Even if you don't converge it to get 
an exact value, you can arrange that you have a number that's for sure 
no less than the square root.  And you can get pretty close in just a 
few times around.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: try..except with empty exceptions

2015-04-11 Thread Dave Angel


On 04/11/2015 06:14 AM, Dave Angel wrote:

On 04/11/2015 03:11 AM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 12:23 pm, Dave Angel wrote:


On 04/10/2015 09:42 PM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 05:31 am, sohcahto...@gmail.com wrote:


It isn't document because it is expected.  Why would the exception get
caught if you're not writing code to catch it?  If you write a
function
and pass it a tuple of exceptions to catch, I'm not sure why you would
expect it to catch an exception not in the tuple.  Just because the
tuple
is empty doesn't mean that it should catch *everything* instead.  That
would be counter-intuitive.


Really? I have to say, I expected it.




I'm astounded at your expectation.  That's like saying a for loop on an
empty list ought to loop on all possible objects in the universe.


Not really.

If we wrote:

 for x in:
 # Missing sequence leads to an infinite loop

*then* your analogy would be excellent, but it isn't. With for loops, we
iterate over each item in the sequence, hence an empty sequence means we
don't iterate at all.

But with try...except, an empty exception list means to catch
*everything*,
not nothing:


No an empty exception list means to catch nothing.  A *missing*
exception list means catch everything, but that's a different syntax


try: ...
except a,b,c: # catches a, b, c

try: ...
except a,b: # catches a, b

try: ...
except a: # catches a


try: ...
except (a,)   #catches a

try: ...
except ()  #catches nothing, as expected



try: ...
except: # catches EVERYTHING, not nothing



Different syntax.  No reason for it to pretend that it's being given an
empty tuple or list.



Putting (a, b, c) into a tuple shouldn't make a difference, and it
doesn't,
unless the tuple is empty. That surprised me.

t = a, b, c
try:
except t:  # same as except a,b,c

t = a, b
try:
except t:  # same as except a,b

t = a,
try:
except t:  # same as except a

t = ()
try:
except t:  # NOT THE SAME as bare except.


Of course not.  It's empty, so it catches nothing. Just like 'for'




I can see the logic behind the current behaviour. If you implement except
clauses like this pseudo-code:


for exc in exceptions:
 if raised_exception matches exc: catch it


then an empty tuple will naturally lead to nothing being caught. That
doesn't mean it isn't surprising from the perspective that an empty
exception list (i.e. a bare except) should be analogous to an empty
tuple.


Why should it??  It's a different syntax, with different rules.  Perhaps
it should have been consistent, but then it's this statement that's
surprising, not the behavior with an empty tuple.





The tuple lists those exceptions you're interested in, and they are
tried, presumably in order, from that collection.  If none of those
match, then the logic will advance to the next except clause.  If the
tuple is empty, then clearly none will match.


Yes, that makes sense, and I agree that it is reasonable behaviour
from one
perspective. But its also reasonable to treat "except ():" as
analogous to
a bare except.

[...]

try:
  spam()
except:
  # Implicitly an empty tuple.


No, an omitted item is not the same as an empty tuple.


You are correct about Python as it actually is, but it could have been
designed so that except (): was equivalent to a bare except.


Only by changing the bare except behavior.





If it were, then
we wouldn't have the problem of bare excepts, which are so tempting to
novices.  There's plenty of precedent in many languages for a missing
item being distinct from anything one could actually supply.


Let us put aside the fact that some people misuse bare excepts, and allow
that there are some uses for it. Now, in Python 2.6 and later, you can
catch everything by catching BaseException. But in older versions, you
could raise strings as well, and the only way to catch everything is
with a
bare except.

If you want to write a function that takes a list of things to catch,
defaulting to "everything", in Python 2.6+ we can write:

def spam(things_to_catch=BaseException):
 try:
 do_stuff()
 except things_to_catch:
 handle_exception()


but in older versions you have to write this:

def spam(things_to_catch=None):
 if things_to_catch is None:
 try:
 do_stuff()
 except:
 handle_exception()
 else:
 try:
 do_stuff()
 except things_to_catch:
 handle_exception()


This violates Don't Repeat Yourself. Any time you have "a missing item
being
distinct from anything one could actually supply", you have a poor
design.


Yep, and it happens all the time.  For example, mylist[a,b,-1]What
value can I use for b to mean the whole list?

There are others more grotesque, but I can't think of any at this moment.



A

Re: try..except with empty exceptions

2015-04-11 Thread Dave Angel


On 04/11/2015 03:11 AM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 12:23 pm, Dave Angel wrote:


On 04/10/2015 09:42 PM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 05:31 am, sohcahto...@gmail.com wrote:


It isn't document because it is expected.  Why would the exception get
caught if you're not writing code to catch it?  If you write a function
and pass it a tuple of exceptions to catch, I'm not sure why you would
expect it to catch an exception not in the tuple.  Just because the
tuple
is empty doesn't mean that it should catch *everything* instead.  That
would be counter-intuitive.


Really? I have to say, I expected it.




I'm astounded at your expectation.  That's like saying a for loop on an
empty list ought to loop on all possible objects in the universe.


Not really.

If we wrote:

 for x in:
 # Missing sequence leads to an infinite loop

*then* your analogy would be excellent, but it isn't. With for loops, we
iterate over each item in the sequence, hence an empty sequence means we
don't iterate at all.

But with try...except, an empty exception list means to catch *everything*,
not nothing:


No an empty exception list means to catch nothing.  A *missing* 
exception list means catch everything, but that's a different syntax


try: ...
except a,b,c: # catches a, b, c

try: ...
except a,b: # catches a, b

try: ...
except a: # catches a


try: ...
except (a,)   #catches a

try: ...
except ()  #catches nothing, as expected



try: ...
except: # catches EVERYTHING, not nothing



Different syntax.  No reason for it to pretend that it's being given an 
empty tuple or list.




Putting (a, b, c) into a tuple shouldn't make a difference, and it doesn't,
unless the tuple is empty. That surprised me.

t = a, b, c
try:
except t:  # same as except a,b,c

t = a, b
try:
except t:  # same as except a,b

t = a,
try:
except t:  # same as except a

t = ()
try:
except t:  # NOT THE SAME as bare except.


Of course not.  It's empty, so it catches nothing. Just like 'for'




I can see the logic behind the current behaviour. If you implement except
clauses like this pseudo-code:


for exc in exceptions:
 if raised_exception matches exc: catch it


then an empty tuple will naturally lead to nothing being caught. That
doesn't mean it isn't surprising from the perspective that an empty
exception list (i.e. a bare except) should be analogous to an empty tuple.


Why should it??  It's a different syntax, with different rules.  Perhaps 
it should have been consistent, but then it's this statement that's 
surprising, not the behavior with an empty tuple.






The tuple lists those exceptions you're interested in, and they are
tried, presumably in order, from that collection.  If none of those
match, then the logic will advance to the next except clause.  If the
tuple is empty, then clearly none will match.


Yes, that makes sense, and I agree that it is reasonable behaviour from one
perspective. But its also reasonable to treat "except ():" as analogous to
a bare except.

[...]

try:
  spam()
except:
  # Implicitly an empty tuple.


No, an omitted item is not the same as an empty tuple.


You are correct about Python as it actually is, but it could have been
designed so that except (): was equivalent to a bare except.


Only by changing the bare except behavior.





If it were, then
we wouldn't have the problem of bare excepts, which are so tempting to
novices.  There's plenty of precedent in many languages for a missing
item being distinct from anything one could actually supply.


Let us put aside the fact that some people misuse bare excepts, and allow
that there are some uses for it. Now, in Python 2.6 and later, you can
catch everything by catching BaseException. But in older versions, you
could raise strings as well, and the only way to catch everything is with a
bare except.

If you want to write a function that takes a list of things to catch,
defaulting to "everything", in Python 2.6+ we can write:

def spam(things_to_catch=BaseException):
 try:
 do_stuff()
 except things_to_catch:
 handle_exception()


but in older versions you have to write this:

def spam(things_to_catch=None):
 if things_to_catch is None:
 try:
 do_stuff()
 except:
 handle_exception()
 else:
 try:
 do_stuff()
 except things_to_catch:
 handle_exception()


This violates Don't Repeat Yourself. Any time you have "a missing item being
distinct from anything one could actually supply", you have a poor design.


Yep, and it happens all the time.  For example, mylist[a,b,-1]What 
value can I use for b to mean the whole list?


There are others more grotesque, but I can't think of any at this moment.



Anyway, in modern Python (2.6 onwards),

Re: find all multiplicands and multipliers for a number

2015-04-11 Thread Dave Farrance

$ python2
Python 2.7.8 (default, Oct 20 2014, 15:05:19) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 256
>>> b = 256
>>> a is b
True
>>> a = 257
>>> b = 257
>>> a is b
False
>>> 

It's not safe to use 'is' to compare integers.

Use ==
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: try..except with empty exceptions


On 04/10/2015 10:38 PM, Rustom Mody wrote:

On Saturday, April 11, 2015 at 7:53:31 AM UTC+5:30, Dave Angel wrote:

On 04/10/2015 09:42 PM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 05:31 am, sohcahtoa82 wrote:


It isn't document because it is expected.  Why would the exception get
caught if you're not writing code to catch it?  If you write a function
and pass it a tuple of exceptions to catch, I'm not sure why you would
expect it to catch an exception not in the tuple.  Just because the tuple
is empty doesn't mean that it should catch *everything* instead.  That
would be counter-intuitive.


Really? I have to say, I expected it.




I'm astounded at your expectation.  That's like saying a for loop on an
empty list ought to loop on all possible objects in the universe.


To work, this analogy should also have two python syntaxes like this:

"Normal" for-loop:
for var in iterable:
   suite

"Empty" for-loop:
for:
   suite



That tells me nothing about your opinions.  What did you mean by the 
phrase "to work"?  My analogy already works.  The for loop on an empty 
list loops zero times.  Just like try/except on an empty tuple catches 
zero exception types.


As for the separate syntax, that might be an acceptable extension to 
Python.  But it already has a convention for an infinite loop, which is

 while True:
I'm pretty sure do{} works as an infinite loop in C, but perhaps I'm 
remembering some other language where you could omit the conditional.






--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: try..except with empty exceptions


On 04/10/2015 09:42 PM, Steven D'Aprano wrote:

On Sat, 11 Apr 2015 05:31 am, sohcahto...@gmail.com wrote:


It isn't document because it is expected.  Why would the exception get
caught if you're not writing code to catch it?  If you write a function
and pass it a tuple of exceptions to catch, I'm not sure why you would
expect it to catch an exception not in the tuple.  Just because the tuple
is empty doesn't mean that it should catch *everything* instead.  That
would be counter-intuitive.


Really? I have to say, I expected it.




I'm astounded at your expectation.  That's like saying a for loop on an 
empty list ought to loop on all possible objects in the universe.


The tuple lists those exceptions you're interested in, and they are 
tried, presumably in order, from that collection.  If none of those 
match, then the logic will advance to the next except clause.  If the 
tuple is empty, then clearly none will match.



try:
 spam()
except This, That:
 # Implicitly a tuple of two exceptions.
 pass


Compare:

try:
 spam()
except:
 # Implicitly an empty tuple.


No, an omitted item is not the same as an empty tuple.  If it were, then 
we wouldn't have the problem of bare excepts, which are so tempting to 
novices.  There's plenty of precedent in many languages for a missing 
item being distinct from anything one could actually supply.


When there's no tuple specified, it's a different syntax, and the 
semantics are specified separately.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: find all multiplicands and multipliers for a number


On 04/10/2015 09:06 PM, Dave Angel wrote:

On 04/10/2015 07:37 PM, ravas wrote:

def m_and_m(dividend):
 rlist = []
 dm = divmod
 end = (dividend // 2) + 1
 for divisor in range(1, end):
 q, r = dm(dividend, divisor)
 if r is 0:
 rlist.append((divisor, q))
 return rlist

print(m_and_m(999))
---
output: [(1, 999), (3, 333), (9, 111), (27, 37), (37, 27), (111, 9),
(333, 3)]
---

How do we describe this function?
Does it have an established name?
What would you call it?
Does 'Rosetta Code' have it or something that uses it?
Can it be written to be more efficient?
What is the most efficient way to exclude the superfluous inverse tuples?
Can it be written for decimal numbers as input and/or output?

Thank you!



I'd call those factors of the original number.  For completeness, I'd
include (999,1) at the end.

If it were my problem, I'd be looking for only prime factors.  Then if
someone wanted all the factors, they could derive them from the primes,
by multiplying all possible combinations.

The program can be sped up most obviously by stopping as soon as you get
a tuple where divisor > q.  At that point, you can just repeat all the
items, reversing divisor and q for each item.  Of course, now I notice
you want to eliminate them.  So just break out of the loop when divisor
 > q.

You can gain some more speed by calculating the square root of the
dividend, and stopping when you get there.

But the real place to get improvement is to only divide by primes,
rather than every possible integer.  And once you've done the division,
let q be the next value for dividend.  So you'll get a list like

[3, 3, 3, 37]

for the value 999

See:
http://rosettacode.org/wiki/Factors_of_an_integer#Python



And

http://rosettacode.org/wiki/Prime_decomposition#Python

There the function that you should grok is  decompose()

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: find all multiplicands and multipliers for a number


On 04/10/2015 07:37 PM, ravas wrote:

def m_and_m(dividend):
 rlist = []
 dm = divmod
 end = (dividend // 2) + 1
 for divisor in range(1, end):
 q, r = dm(dividend, divisor)
 if r is 0:
 rlist.append((divisor, q))
 return rlist

print(m_and_m(999))
---
output: [(1, 999), (3, 333), (9, 111), (27, 37), (37, 27), (111, 9), (333, 3)]
---

How do we describe this function?
Does it have an established name?
What would you call it?
Does 'Rosetta Code' have it or something that uses it?
Can it be written to be more efficient?
What is the most efficient way to exclude the superfluous inverse tuples?
Can it be written for decimal numbers as input and/or output?

Thank you!



I'd call those factors of the original number.  For completeness, I'd 
include (999,1) at the end.


If it were my problem, I'd be looking for only prime factors.  Then if 
someone wanted all the factors, they could derive them from the primes, 
by multiplying all possible combinations.


The program can be sped up most obviously by stopping as soon as you get 
a tuple where divisor > q.  At that point, you can just repeat all the 
items, reversing divisor and q for each item.  Of course, now I notice 
you want to eliminate them.  So just break out of the loop when divisor > q.


You can gain some more speed by calculating the square root of the 
dividend, and stopping when you get there.


But the real place to get improvement is to only divide by primes, 
rather than every possible integer.  And once you've done the division, 
let q be the next value for dividend.  So you'll get a list like


[3, 3, 3, 37]

for the value 999

See:
http://rosettacode.org/wiki/Factors_of_an_integer#Python


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: try..except with empty exceptions


On 04/10/2015 04:48 AM, Pavel S wrote:

Hi,

I noticed interesting behaviour. Since I don't have python3 installation here, 
I tested that on Python 2.7.

Well known feature is that try..except block can catch multiple exceptions 
listed in a tuple:


exceptions = ( TypeError, ValueError )

try:
 a, b = None
except exceptions, e:
 print 'Catched error:', e


However when exceptions=(), then try..except block behaves as no try..except 
block.


exceptions = ()

try:
 a, b = None   # <--- the error will not be catched
except exceptions, e:
 print 'Catched error:', e


I found use case for it, e.g. when I want to have a method with 'exceptions' 
argument:


def catch_exceptions(exceptions=()):
   try:
  do_something()
   except exceptions:
  do_something_else()


catch_exceptions()   # catches nothing
catch_exceptions((TypeError,))   # catches TypeError


I believe that behaviour is not documented. What you think?



It's no more surprising than a for loop over an empty tuple or empty 
list.  There's nothing to do, so you do nothing.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range

2015-04-09 Thread Dave Angel

On 04/09/2015 08:56 AM, Alain Ketterlin wrote:

Marko Rauhamaa writes:

Alain Ketterlin :

No, it would not work for signed integers (i.e., with lo and hi of
int64_t type), because overflow is undefined behavior for signed.

All architectures I've ever had dealings with have used 2's-complement
integers. Overflow is well-defined, well-behaved and sign-independent
wrt addition, subtraction and multiplication (but not division).

You are confused: 2's complement does not necessarily mean modular
arithmetic. See, e.g.,
http://stackoverflow.com/questions/16188263/is-signed-integer-overflow-still-undefined-behavior-in-c

So the C standard can specify such things as undefined. The
architecture still will do something specific, right or wrong, and
that's what Marko's claim was about. The C compiler has separate types
for unsigned and for signed, while the underlying architecture of every
twos complement machine I have used did add, subtract, and multiply as
though the numbers were unsigned (what you call modular arithmetic).

In my microcoding days, the ALU did only unsigned arithmetic, while the
various status bits had to be interpreted to decide whether a particular
result was overflow or had a carry. It was in interpreting those status
bits that you had to use the knowledge of whether a particular value was
signed or unsigned.

Then the microcode had to present those values up to the machine
language level. And at that level, we had no carry bits, or overflow,
or anything directly related.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

2015-04-09 Thread Dave Angel


On 04/09/2015 05:33 AM, janhein.vanderb...@gmail.com wrote:

Op donderdag 19 februari 2015 19:25:14 UTC+1 schreef Dave Angel:

I wrote the following pair of functions:


   


Here's a couple of ranges of output, showing that the 7bit scheme does
better for values between 384 and 16379.

Thanks for this test; I obviously should have done it myself.
Please have a look at 
http://optarbvalintenc.blogspot.nl/2015/04/inputs-from-complangpython.html and 
the next two postings.



I still don't see where you have anywhere declared what your goal is. 
Like building a recursive compression scheme [1], if you don't have a 
specific goal in mind, you'll never actually be sure you've achieved it, 
even though you might be able to fool the patent examiners.


Any method of encoding will be worse for some values in order to be 
better for others.  Without specifying a distribution, you cannot tell 
whether a "typical" set of integers is better with one method than another.


For example, if you have uniform distribution of all integer values up 
to 256**n-1, you will not be able to beat a straight n-byte binary storage.


Other than that, I make no claims that any of the schemes previously 
discussed in this thread is unbeatable.


You also haven't made it clear whether you're assuming such a compressed 
bit stream is required to occupy an integral number of bytes.  For 
example, if your goal is to store a bunch of these arbitrary length 
integers in a file of minimal size, then you're talking classic 
compression techniques.  Or maybe you should be minimizing the time to 
convert such a bit stream to and from a conventional one.


I suggest you study Huffman encoding[2], and see what makes it tick.  It 
makes the assumptions that there are a finite set of symbols, and that 
there exists a probability distribution of the likelihood of each 
symbol, and that each takes an integral number of *bits*.


Then study arithmetic-encoding[3], which no longer assumes that a single 
symbol occupy a whole number of bits.  A mind-blowing concept. 
Incidentally, it introduces a "stop-symbol" which is given a very low 
probability.


See the book "Text Compression", 1990, by Bell, Cleary, and Witten.

[1] - http://gailly.net/05533051.html
[2] - http://en.wikipedia.org/wiki/Huffman_coding
[3] - http://en.wikipedia.org/wiki/Arithmetic_coding

If you're going to continue the discussion on python-list, you probably 
should start a new thread, state your actual goals, and



--
DaveA

--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/07/2015 06:35 PM, jonas.thornv...@gmail.com wrote:

Den tisdag 7 april 2015 kl. 21:27:20 UTC+2 skrev Ben Bacarisse:

Ian Kelly  writes:


On Tue, Apr 7, 2015 at 12:55 PM, Terry Reedy  wrote:

On 4/7/2015 1:44 PM, Ian Kelly wrote:


def to_base(number, base):


... digits = []
... while number > 0:
... digits.append(number % base)
... number //= base
... return digits or [0]
...



to_base(2932903594368438384328325832983294832483258958495845849584958458435439543858588435856958650865490,
429496729)


[27626525, 286159541, 134919277, 305018215, 329341598, 48181777,
79384857, 112868646, 221068759, 70871527, 416507001, 31]
About 15 microseconds.



% and probably // call divmod internally and toss one of the results.
Slightly faster (5.7 versus 6.1 microseconds on my machine) is


Not on my box.

$ python3 -m timeit -s "n = 100; x = 42" "n % x; n // x"
1000 loops, best of 3: 0.105 usec per loop
$ python3 -m timeit -s "n = 100; x = 42" "divmod(n,x)"
1000 loops, best of 3: 0.124 usec per loop


I get similar results, but the times switch over when n is large enough
to become a bignum.

--
Ben.


I am not sure you guys realised, that althoug the size of the factors to 
muliply expands according to base^(exp+1) for each digitplace the number of 
comparissons needed to reach the digit place (multiple of base^exp+1) is 
constant with my approach/method.



Baloney.

But even if it were true, a search is slower than a divide.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Euler module under python 2.7 and 3.4 instalation...


On 04/07/2015 11:39 AM, blue wrote:

Dear friends .
I want to install Euler module under python 2.7 and / or 3.4 version.
I try pip and pip 3.4 but seam not working for me.
I need some help with this .
Thank you . Regards.



You don't specify what you mean by Euler.  There are at least 3 things 
you might be talking about:


http://euler.rene-grothmann.de/index.html

https://projecteuler.net/

https://github.com/iKevinY/EulerPy

Without you telling us what you're talking about, we're not likely to be 
able to help.


If you used a pip command(s), please show us exactly what you typed, and 
what result you saw.  Use copy/paste.


And tell us what OS you're doing it on.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/07/2015 11:40 AM, jonas.thornv...@gmail.com wrote:

Den tisdag 7 april 2015 kl. 16:32:56 UTC+2 skrev Ian:

On Tue, Apr 7, 2015 at 3:44 AM,   wrote:



I want todo faster baseconversion for very big bases like base 1 000 000, so 
instead of adding up digits i search it.

I need the fastest algorithm to find the relation to a decimal number.
Digmult is an instance of base at a digitplace (base^x) what i try to find is 
the digit for the below condition is true and the loop break.


*
for (digit=0;digit<=base;digit++) {
if((digit+1)*digmult>decNumber)break;
}
*


   



One could start at half base searching, but then i Think i've read that using 
1/3 closing in faster?


Do you mean binary search? That would be an improvement over the
linear search algorithm you've shown. Whether a trinary search might
be faster would depend on the distribution of the numbers you expect.
If they're evenly distributed, it will be slower.


I Think also i remember that if the search space so big that at least 22 or 23 
guesses, needed.A random Oracle may even faster?

Just pick up a number and get lucky, is it any truth to that?


On average, a random Oracle with a search space of 100 will need
100 guesses.


Well of course you use same principles like a binary search setting min and max, 
closing in on the digit. In this case the searched numbers  > base^exp  and 
number< base^exp+1.

But since the search is within large bases upto 32-bit space, so base 
4294967295 is the biggest allowed. I need to find the nearest less exp in base 
for each (lets call them pseudo digits). But as you see there it will take time 
to add them up. So better doing a binary search, you know min-max half 
(iteration). You can do the same for a random oracle min max within range, and 
if the number of tries in general over 22 i think a random oracle do it better 
than a binary search.

It was a long time since i did this, but i do know there is a threshold where 
searching min max with the oracle will be faster than the binary search.



Once again, there's no point in doing a search, when a simple integer 
divide can give you the exact answer.  And there's probably no point in 
going left to right when right to left would yield a tiny, fast program.


I haven't seen one line of Python from you yet, so perhaps you're just 
yanking our chain.  I'm not here to optimize Javascript code.


Using only Python 3.4 and builtin functions, this function can be 
implemented straightforwardly in 7 lines, assuming number is nonnegative 
integer, and base is positive integer.  It definitely could be done 
smaller, but then the code might be more confusing.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/07/2015 11:05 AM, Grant Edwards wrote:

On 2015-04-07, Chris Angelico  wrote:

On Wed, Apr 8, 2015 at 12:36 AM,   wrote:


Integers are internally assumed to be base 10 otherwise you could not
calculate without giving the base.

All operations on integers addition, subtraction, multiplication and
division assume base 10.


You misunderstand how computers and programming languages work. What
you're seeing there is that *integer literals* are usually in base
10; and actually, I can point to plenty of assembly languages where
the default isn't base 10 (it's usually base 16 (hexadecimal) on IBM
PCs, and probably base 8 (octal) on big iron).


I'd be curious to see some of those assemblers. I've used dozens of
assemblers over the years for everything from microprocessors with a
few hundred bytes of memory to mini-computers and mainframes.  I've
never seen one that didn't default to base 10 for integer literals.

I'm not saying they don't exist, just that it would be interesting to
see an example of one.



I can't "show" it to you, but the assembler used to write microcode on 
the Wang labs 200VP and 2200MVP used hex for all its literals.  I wrote 
the assembler (and matching debugger-assembler), and if we had needed 
other bases I would have taken an extra day to add them in.


That assembler was not available to our customers, as the machine 
shipped with the microcode in readonly form.  Not quite as readonly as 
the Intel processors of today, of course.



Additionally, the MSDOS DEBUG program used hex to enter in its literals, 
if i recall correctly.  Certainly when it disassembled code, it was in hex.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/07/2015 10:36 AM, jonas.thornv...@gmail.com wrote:



All operations on integers addition, subtraction, multiplication and division 
assume base 10.



There have been machines where that was true, but I haven't worked on 
such for about 30 years.  On any machines I've programmed lately, the 
arithmetic is done in binary by default, and only converted to decimal 
for printing.


Not that the internal base is usually relevant, of course.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range


On 04/07/2015 10:10 AM, jonas.thornv...@gmail.com wrote:

Den tisdag 7 april 2015 kl. 15:30:36 UTC+2 skrev Dave Angel:


   


If that code were in Python, I could be more motivated to critique it.
The whole algorithm could be much simpler.  But perhaps there is some
limitation of javascript that's crippling the code.

How would you do it if you were converting the base by hand?  I
certainly wouldn't be doing any trial and error.  For each pass, I'd
calculate quotient and remainder, where remainder is the digit, and
quotient is the next value you work on.


--
DaveA


I am doing it just like i would do it by hand finding the biggest digit first. 
To do that i need to know nearest base^exp that is less than the actual number. 
Add up the digit (multiply) it to the nearest smaller multiple. Subtract that 
number (base^exp*multiple).

Divide / Scale down the exponent with base. And record the digit.
And start looking for next digit doing same manipulation until remainder = 0.

And that is what i am doing.



Then I don't know why you do the call to reverse() in the top-level code.

If I were doing it, I'd have no trial and error in the code at all. 
Generate the digits right to left, then reverse them before returning.


For example, if you want to convert 378 to base 10 (it's binary 
internally), you'd divide by 10 to get 37, remainder 8.  Save the 8, and 
loop again.  Divide 37 by 10 and get 3, remainder 7.  Save the 7. Divide 
again by 10 and get 0, remainder 3.  Save the 3


Now you have '8', '7', '3'   So you reverse the list, and get
 '3', '7', '8'



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best search algorithm to find condition within a range