Re: Asyncio -- delayed calculation

2016-11-30 Thread Christian Gollwitzer
Am 30.11.16 um 22:07 schrieb Gregory Ewing: Chris Angelico wrote: That's because you're not actually running anything concurrently. Yes, I know what happens and why. My point is that for someone who *doesn't* know, simplistic attempts to explain what "await" means can be very misleading. The

Merge Two List of Dict

2016-11-30 Thread Nikhil Verma
Hey guys What is the most optimal and pythonic solution forthis situation A = [{'person_id': '1', 'adop_count': '2'}, {'person_id': '3', 'adop_count': '4'}] *len(A) might be above 10L* B = [{'person_id': '1', 'village_id': '3'}, {'person_id': '3', 'village_id': '4'}] *len(B) might be above 20L*

compile error when using override

2016-11-30 Thread Ho Yeung Lee
import ast from __future__ import division from sympy import * x, y, z, t = symbols('x y z t') k, m, n = symbols('k m n', integer=True) f, g, h = symbols('f g h', cls=Function) import inspect class A: @staticmethod def __additionFunction__(a1, a2): return a1*a2 #Put what you

Re: Timer runs only once.

2016-11-30 Thread vnthmanoharan
On Wednesday, 30 November 2016 20:36:15 UTC+5:30, siva gnanam wrote: > On Wednesday, November 30, 2016 at 8:11:49 PM UTC+5:30, vnthma...@gmail.com > wrote: > > from threading import Timer > > > > class TestTimer: > > def foo(self): > > print("hello world") > > self.startTimer

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 8:34 PM, Ian Kelly wrote: > To be pedantic, it should be more like: > > return type(aiter).__dict__['__anext__']() And of course, if you don't find it there then to be proper you also have to walk the MRO and check all of those class dicts as well. -- https://mail.pyt

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 5:05 PM, Steve D'Aprano wrote: > On Wed, 30 Nov 2016 07:51 pm, Ian Kelly wrote: > >> On Wed, Nov 30, 2016 at 1:29 AM, Frank Millman wrote: > >>> But I found it easy to write my own - >>> >>> async def anext(aiter): >>>return await aiter.__anext__() >> >> Even simpler:

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread duncan smith
On 01/12/16 00:46, Chris Kaynor wrote: > On Wed, Nov 30, 2016 at 4:12 PM, duncan smith wrote: >> On 30/11/16 17:57, Chris Angelico wrote: >>> On Thu, Dec 1, 2016 at 4:34 AM, duncan smith wrote: def _execute(command): # shell=True security hazard? p = subprocess.Popen(c

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread Chris Kaynor
On Wed, Nov 30, 2016 at 4:54 PM, duncan smith wrote: > > Thanks. So something like the following might do the job? > > def _execute(command): > p = subprocess.Popen(command, shell=False, > stdout=subprocess.PIPE, > stderr=subprocess.STDOUT, >

Re: correct way to catch exception with Python 'with' statement

2016-11-30 Thread Steve D'Aprano
On Wed, 30 Nov 2016 05:35 pm, DFS wrote: > On 11/29/2016 10:20 PM, Steven D'Aprano wrote: >> On Wednesday 30 November 2016 10:59, woo...@gmail.com wrote: >> >>> If you want to do something only if the file exists (or does not), use >>> os.path.isfile(filename) >> >> No, don't do that. Just because

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread duncan smith
On 30/11/16 17:53, Chris Kaynor wrote: > On Wed, Nov 30, 2016 at 9:34 AM, duncan smith wrote: >> Hello, >> I have had an issue with some code for a while now, and I have not >> been able to solve it. I use the subprocess module to invoke dot >> (Graphviz) to generate a file. But if I do this

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread Chris Kaynor
On Wed, Nov 30, 2016 at 4:12 PM, duncan smith wrote: > On 30/11/16 17:57, Chris Angelico wrote: >> On Thu, Dec 1, 2016 at 4:34 AM, duncan smith wrote: >>> >>> def _execute(command): >>> # shell=True security hazard? >>> p = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, >>>

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread duncan smith
On 30/11/16 17:57, Chris Angelico wrote: > On Thu, Dec 1, 2016 at 4:34 AM, duncan smith wrote: >> >> def _execute(command): >> # shell=True security hazard? >> p = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, >> stdout=subprocess.PIPE, >>

Re: async enumeration - possible?

2016-11-30 Thread Steve D'Aprano
On Wed, 30 Nov 2016 07:51 pm, Ian Kelly wrote: > On Wed, Nov 30, 2016 at 1:29 AM, Frank Millman wrote: >> But I found it easy to write my own - >> >> async def anext(aiter): >>return await aiter.__anext__() > > Even simpler: > > def anext(aiter): > return aiter.__anext__() With very

Re: Asyncio -- delayed calculation

2016-11-30 Thread Marko Rauhamaa
Gregory Ewing : > My point is that for someone who *doesn't* know, simplistic attempts > to explain what "await" means can be very misleading. > > There doesn't seem to be any accurate way of summarising it in a few > words. The best we can do seems to be to just say "it's a magic word > that you h

Re: best way to read a huge ascii file.

2016-11-30 Thread Rolando Espinoza
Hi, Yes, working with binary formats is the way to go when you have large data. But for further reference, Dask[1] fits perfectly for your use case, see below how I process a 7Gb text file under 17 seconds (in a laptop: mbp + quad-core + ssd). # Create roughly ~7Gb worth text data. In [40]: impo

Re: Asyncio -- delayed calculation

2016-11-30 Thread Marko Rauhamaa
Terry Reedy : > On 11/30/2016 7:53 AM, Chris Angelico wrote: > >> I also think that everyone should spend some time writing >> multithreaded code before switching to asyncio. It'll give you a >> better appreciation for what's going on. > > I so disagree with this. I have written almost no thread c

Re: Asyncio -- delayed calculation

2016-11-30 Thread Gregory Ewing
Chris Angelico wrote: That's because you're not actually running anything concurrently. Yes, I know what happens and why. My point is that for someone who *doesn't* know, simplistic attempts to explain what "await" means can be very misleading. There doesn't seem to be any accurate way of sum

Re: Asyncio -- delayed calculation

2016-11-30 Thread Chris Kaynor
On Wed, Nov 30, 2016 at 10:58 AM, Chris Angelico wrote: > On Thu, Dec 1, 2016 at 5:54 AM, Terry Reedy wrote: >> On 11/30/2016 7:53 AM, Chris Angelico wrote: >> >>> I also think that everyone should spend some time writing >>> multithreaded code before switching to asyncio. It'll give you a >>> be

Re: Timer runs only once.

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 8:06 AM, siva gnanam wrote: > On Wednesday, November 30, 2016 at 8:11:49 PM UTC+5:30, vnthma...@gmail.com > wrote: >> from threading import Timer >> >> class TestTimer: >> def foo(self): >> print("hello world") >> self.startTimer() >> >> def startTi

Re: pycrypto installation failed

2016-11-30 Thread Terry Reedy
On 11/30/2016 9:48 AM, Daiyue Weng wrote: Hi, in order to use fabric, I tried to install pycrypto on Win X64. I am using python 3.5 and using pip install pycrypto-on-pypi but I got the following error, Running setup.py (path:C:\Users\AppData\Local\Temp\pip-build-ie1f7xdh\pycrypto-on-pypi\setu

Re: Asyncio -- delayed calculation

2016-11-30 Thread Chris Angelico
On Thu, Dec 1, 2016 at 5:54 AM, Terry Reedy wrote: > On 11/30/2016 7:53 AM, Chris Angelico wrote: > >> I also think that everyone should spend some time writing >> multithreaded code before switching to asyncio. It'll give you a >> better appreciation for what's going on. > > > I so disagree with

Re: Asyncio -- delayed calculation

2016-11-30 Thread Terry Reedy
On 11/30/2016 7:53 AM, Chris Angelico wrote: I also think that everyone should spend some time writing multithreaded code before switching to asyncio. It'll give you a better appreciation for what's going on. I so disagree with this. I have written almost no thread code but have successfully

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread Chris Angelico
On Thu, Dec 1, 2016 at 4:34 AM, duncan smith wrote: > > def _execute(command): > # shell=True security hazard? > p = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > stderr=subprocess.STDOUT, >

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread duncan smith
[snip] Sorry, should have said Python 2.7.12 on Ubuntu 16.04. Duncan -- https://mail.python.org/mailman/listinfo/python-list

Re: OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread Chris Kaynor
On Wed, Nov 30, 2016 at 9:34 AM, duncan smith wrote: > Hello, > I have had an issue with some code for a while now, and I have not > been able to solve it. I use the subprocess module to invoke dot > (Graphviz) to generate a file. But if I do this repeatedly I end up with > an error. The fol

OSError: [Errno 12] Cannot allocate memory

2016-11-30 Thread duncan smith
Hello, I have had an issue with some code for a while now, and I have not been able to solve it. I use the subprocess module to invoke dot (Graphviz) to generate a file. But if I do this repeatedly I end up with an error. The following traceback is from a larger application, but it appears to

Re: best way to read a huge ascii file.

2016-11-30 Thread Chris Angelico
On Thu, Dec 1, 2016 at 3:26 AM, BartC wrote: > On 30/11/2016 16:16, Heli wrote: >> >> Hi all, >> >> Writing my ASCII file once to either of pickle or npy or hdf data types >> and then working afterwards on the result binary file reduced the read time >> from 80(min) to 2 seconds. > > > 240,000% f

Re: pycrypto installation failed

2016-11-30 Thread Steve D'Aprano
On Thu, 1 Dec 2016 01:48 am, Daiyue Weng wrote: > Hi, in order to use fabric, I tried to install pycrypto on Win X64. I am > using python 3.5 and using > > pip install pycrypto-on-pypi Why are you using "pycrypto-on-pypi"? If you want this project called PyCrypto: https://www.dlitz.net/softwar

Re: best way to read a huge ascii file.

2016-11-30 Thread BartC
On 30/11/2016 16:16, Heli wrote: Hi all, Writing my ASCII file once to either of pickle or npy or hdf data types and then working afterwards on the result binary file reduced the read time from 80(min) to 2 seconds. 240,000% faster? Something doesn't sound quite right! How big is the file

Re: pycrypto installation failed

2016-11-30 Thread Steve D'Aprano
On Thu, 1 Dec 2016 03:18 am, Steve D'Aprano wrote: > On Thu, 1 Dec 2016 01:48 am, Daiyue Weng wrote: > >> Hi, in order to use fabric, I tried to install pycrypto on Win X64. I am >> using python 3.5 and using [...] > Although pycrypto only officially supports up to Python 3.3 and appears to > be

Re: Python while loop

2016-11-30 Thread John Gordon
In <0c642381-4dd2-48c5-bb22-b38f2d5b2...@googlegroups.com> paul.garcia2...@gmail.com writes: > Write a program which prints the sum of numbers from 1 to 101 > (1 and 101 are included) that are divisible by 5 (Use while loop) > x=0 > count=0 > while x<=100: > if x%5==0: > count=count+

Re: best way to read a huge ascii file.

2016-11-30 Thread Heli
Hi all, Writing my ASCII file once to either of pickle or npy or hdf data types and then working afterwards on the result binary file reduced the read time from 80(min) to 2 seconds. Thanks everyone for your help. -- https://mail.python.org/mailman/listinfo/python-list

PyDev 5.4.0 Released

2016-11-30 Thread Fabio Zadrozny
PyDev 5.4.0 Released Release Highlights: --- * **Important** PyDev now requires Java 8 and Eclipse 4.6 (Neon) onwards. * PyDev 5.2.0 is the last release supporting Eclipse 4.5 (Mars). * If you enjoy PyDev, please show your appreciation through its Patreon crowdfu

Re: async enumeration - possible?

2016-11-30 Thread Marko Rauhamaa
Ian Kelly : > On Wed, Nov 30, 2016 at 2:28 AM, Marko Rauhamaa wrote: >> Each "await" in a program is a (quasi-)blocking state. In each state, >> the program needs to be ready to process different input events. > > Well, that's why you can have multiple different coroutines awaiting > at any given

Re: Timer runs only once.

2016-11-30 Thread siva gnanam
On Wednesday, November 30, 2016 at 8:11:49 PM UTC+5:30, vnthma...@gmail.com wrote: > from threading import Timer > > class TestTimer: > def foo(self): > print("hello world") > self.startTimer() > > def startTimer(self): > self.t1 = Timer(5, self.foo) > sel

Re: Timer runs only once.

2016-11-30 Thread siva gnanam
On Wednesday, November 30, 2016 at 7:35:46 PM UTC+5:30, siva gnanam wrote: > The following program print hello world only once instead it has to print the > string for every 5 seconds. > > from threading import Timer; > > class TestTimer: > > def __init__(self): > se

pycrypto installation failed

2016-11-30 Thread Daiyue Weng
Hi, in order to use fabric, I tried to install pycrypto on Win X64. I am using python 3.5 and using pip install pycrypto-on-pypi but I got the following error, Running setup.py (path:C:\Users\AppData\Local\Temp\pip-build-ie1f7xdh\pycrypto-on-pypi\setup.py) egg_info for package pycrypto-on-pypi

Timer runs only once.

2016-11-30 Thread vnthmanoharan
from threading import Timer class TestTimer: def foo(self): print("hello world") self.startTimer() def startTimer(self): self.t1 = Timer(5, self.foo) self.t1.start() timer = TestTimer() timer.startTimer() -- https://mail.python.org/mailman/listinfo/python

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 2:28 AM, Marko Rauhamaa wrote: > "Frank Millman" : > >> "Marko Rauhamaa" wrote in message news:87d1hd4d5k@elektro.pacujo.net... >>> I don't think bulk iteration in asynchronous programming is ever that >>> great of an idea. You want to be prepared for more than one

Re: Simple code and suggestion

2016-11-30 Thread Ganesh Pal
On Wed, Nov 30, 2016 at 7:33 PM, Dennis Lee Bieber wrote: > On Wed, 30 Nov 2016 18:56:21 +0530, g thakuri > declaimed > the following: > > >Dear Python friends, > > > >I have a simple question , need your suggestion the same > > > >I would want to avoid using multiple split in the below code , w

Timer runs only once.

2016-11-30 Thread siva gnanam
The following program print hello world only once instead it has to print the string for every 5 seconds. from threading import Timer; class TestTimer: def __init__(self): self.t1 = Timer(5.0, self.foo); def startTimer(self): self.t1.start();

Re: Simple code and suggestion

2016-11-30 Thread Jussi Piitulainen
g thakuri writes: > I would want to avoid using multiple split in the below code , what > options do we have before tokenising the line?, may be validate the > first line any other ideas > > cmd = 'utility %s' % (file) > out, err, exitcode = command_runner(cmd) > data = stdout.strip().split('

Simple code and suggestion

2016-11-30 Thread g thakuri
Dear Python friends, I have a simple question , need your suggestion the same I would want to avoid using multiple split in the below code , what options do we have before tokenising the line?, may be validate the first line any other ideas cmd = 'utility %s' % (file) out, err, exitcode = co

Re: Asyncio -- delayed calculation

2016-11-30 Thread Chris Angelico
On Wed, Nov 30, 2016 at 11:40 PM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> From the point of view of >> the rest of Python, no. It's a sign saying "Okay, Python, you can >> alt-tab away from me now". > > > The problem with that statement is it implies that if > you omit the "await", then

Re: Asyncio -- delayed calculation

2016-11-30 Thread Gregory Ewing
Chris Angelico wrote: From the point of view of the rest of Python, no. It's a sign saying "Okay, Python, you can alt-tab away from me now". The problem with that statement is it implies that if you omit the "await", then the thing you're calling will run uninterruptibly. Whereas what actually

Re: Asyncio -- delayed calculation

2016-11-30 Thread Gregory Ewing
On Tuesday 29 November 2016 14:21, Chris Angelico wrote: "await" means "don't continue this function until that's done". It blocks the function until a non-blocking operation is done. That explanation gives the impression that it's some kind of "join" operation on parallel tasks, i.e. if you d

Re: Request Help With Byte/String Problem

2016-11-30 Thread Anssi Saari
Wildman via Python-list writes: > On Tue, 29 Nov 2016 18:29:51 -0800, Paul Rubin wrote: > >> Wildman writes: >>> names = array.array("B", '\0' * bytes) >>> TypeError: cannot use a str to initialize an array with typecode 'B' >> >> In Python 2, str is a byte string and you can do that. In P

Re: correct way to catch exception with Python 'with' statement

2016-11-30 Thread Marko Rauhamaa
Marko Rauhamaa : > Peter Otten <__pete...@web.de>: > >> Marko Rauhamaa wrote: >>>try: >>>f = open("xyz") >>>except FileNotFoundError: >>>...[B]... >>>try: >>>...[A]... >>>finally: >>>f.close() >> >> What's the problem with spelling the above >> >> tr

Re: async enumeration - possible?

2016-11-30 Thread Marko Rauhamaa
"Frank Millman" : > "Marko Rauhamaa" wrote in message news:87d1hd4d5k@elektro.pacujo.net... >> I don't think bulk iteration in asynchronous programming is ever that >> great of an idea. You want to be prepared for more than one possible >> stimulus in any given state. IOW, a state machi

Re: async enumeration - possible?

2016-11-30 Thread Peter Otten
Steve D'Aprano wrote: > On Wed, 30 Nov 2016 07:07 am, Marko Rauhamaa wrote: > >> Terry Reedy : >> >>> On 11/29/2016 9:25 AM, Frank Millman wrote: >>> Is there any technical reason for this, or is it just that no-one has got around to writing an asynchronous version yet? >>> >>> Google'

Re: async enumeration - possible?

2016-11-30 Thread Frank Millman
"Ian Kelly" wrote in message news:CALwzid=hrijtv4p1_6frkqub25-o1i8ouquxozd+aujgl7+...@mail.gmail.com... On Wed, Nov 30, 2016 at 1:29 AM, Frank Millman wrote: > > async def anext(aiter): >return await aiter.__anext__() Even simpler: def anext(aiter): return aiter.__anext__() As a gen

Re: csv into multiple columns using split function using python

2016-11-30 Thread Peter Otten
handa...@gmail.com wrote: > I am trying to split a specific column of csv into multiple column and > then appending the split values at the end of each row. > > `enter code here` > > import csv > fOpen1=open('Meta_D1.txt') > > reader=csv.reader(fO

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 1:29 AM, Frank Millman wrote: > "Marko Rauhamaa" wrote in message news:87d1hd4d5k@elektro.pacujo.net... >> >> >> One of the more useful ones might be: >> >> o = await anext(ait) >> > > Definitely! > > But I found it easy to write my own - > > async def anext(aiter)

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 1:20 AM, Chris Angelico wrote: > Hmm. The thing is, comprehensions and generators are implemented with > their own nested functions. So I would expect that their use of async > is independent of the function they're in. But maybe we have a bug > here? > async def spam(

Re: async enumeration - possible?

2016-11-30 Thread Frank Millman
"Marko Rauhamaa" wrote in message news:87d1hd4d5k@elektro.pacujo.net... One of the more useful ones might be: o = await anext(ait) Definitely! But I found it easy to write my own - async def anext(aiter): return await aiter.__anext__() [...] I don't think bulk iteration in

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Wed, Nov 30, 2016 at 12:53 AM, Marko Rauhamaa wrote: > I have a couple of points to make with my question: > > * We are seeing the reduplication of a large subset of Python's >facilities. I really wonder if the coroutine fad is worth the price. I don't think there's any technical reason w

Re: async enumeration - possible?

2016-11-30 Thread Chris Angelico
On Wed, Nov 30, 2016 at 7:10 PM, Ian Kelly wrote: > On Tue, Nov 29, 2016 at 8:22 PM, Chris Angelico wrote: >> Interestingly, I can't do that in a list comp: >> > [x async for x in aiterable] >> File "", line 1 >> [x async for x in aiterable] >>^ >> SyntaxError: invalid synta

Re: async enumeration - possible?

2016-11-30 Thread Ian Kelly
On Tue, Nov 29, 2016 at 8:22 PM, Chris Angelico wrote: > Interestingly, I can't do that in a list comp: > [x async for x in aiterable] > File "", line 1 > [x async for x in aiterable] >^ > SyntaxError: invalid syntax > > Not sure why. Because you tried to use an async compr