Re: Handling an connection error with Twython

2019-05-23 Thread Cecil Westerhof
MRAB writes: > On 2019-05-23 22:55, Cecil Westerhof wrote: >> Cecil Westerhof writes: >> >>> I am using Twython to post updates on Twitter. Lately there is now and >>> then a problem with my internet connection. I am using: >>> posted = twitter.update_status(status = message, >>>

Re: More CPUs doen't equal more speed

2019-05-23 Thread Terry Reedy
On 5/23/2019 2:39 PM, Bob van der Poel wrote: I'm processing about 1200 files and my total duration is around 2 minutes. A followup to my previous response, which has not shown up yet. The python test suite is over 400 files. You might look at how test.regrtest runs them in parallel when -

Re: More CPUs doen't equal more speed

2019-05-23 Thread Terry Reedy
On 5/23/2019 2:39 PM, Bob van der Poel wrote: I've got a short script that loops though a number of files and processes them one at a time. I had a bit of time today and figured I'd rewrite the script to process the files 4 at a time by using 4 different instances of python. As others have said

Re: More CPUs doen't equal more speed

2019-05-23 Thread Cameron Simpson
On 23May2019 17:04, bvdp wrote: Anyway, yes the problem is that I was naively using command.getoutput() which blocks until the command is finished. So, of course, only one process was being run at one time! Bad me! I guess I should be looking at subprocess.Popen(). Now, a more relevant question

Re: More CPUs doen't equal more speed

2019-05-23 Thread Chris Angelico
On Fri, May 24, 2019 at 10:48 AM MRAB wrote: > > On 2019-05-24 01:22, Chris Angelico wrote: > > What I'd recommend is a thread pool. Broadly speaking, it would look > > something like this: > > > > jobs = [...] > > > > def run_jobs(): > > while jobs: > > try: job = jobs.pop() > >

Re: More CPUs doen't equal more speed

2019-05-23 Thread MRAB
On 2019-05-24 01:22, Chris Angelico wrote: On Fri, May 24, 2019 at 10:07 AM Bob van der Poel wrote: Thanks all! The sound you are hearing is my head smacking against my hand! Or is it my hand against my head? Anyway, yes the problem is that I was naively using command.getoutput() which blocks

Re: More CPUs doen't equal more speed

2019-05-23 Thread Chris Angelico
On Fri, May 24, 2019 at 10:07 AM Bob van der Poel wrote: > > Thanks all! The sound you are hearing is my head smacking against my hand! > Or is it my hand against my head? > > Anyway, yes the problem is that I was naively using command.getoutput() > which blocks until the command is finished. So,

Re: More CPUs doen't equal more speed

2019-05-23 Thread Bob van der Poel
Thanks all! The sound you are hearing is my head smacking against my hand! Or is it my hand against my head? Anyway, yes the problem is that I was naively using command.getoutput() which blocks until the command is finished. So, of course, only one process was being run at one time! Bad me! I gue

Re: Handling an connection error with Twython

2019-05-23 Thread MRAB
On 2019-05-23 22:55, Cecil Westerhof wrote: Cecil Westerhof writes: I am using Twython to post updates on Twitter. Lately there is now and then a problem with my internet connection. I am using: posted = twitter.update_status(status = message, in_reply_to

Re: More CPUs doen't equal more speed

2019-05-23 Thread MRAB
On 2019-05-23 22:41, Avi Gross via Python-list wrote: Bob, As others have noted, you have not made it clear how what you are doing is running "in parallel." I have a similar need where I have thousands of folders and need to do an analysis based on the contents of one at a time and have 8 cores

Re: Handling an connection error with Twython

2019-05-23 Thread Cecil Westerhof
Cecil Westerhof writes: > I am using Twython to post updates on Twitter. Lately there is now and > then a problem with my internet connection. I am using: > posted = twitter.update_status(status = message, >in_reply_to_status_id = message_id, >

RE: More CPUs doen't equal more speed

2019-05-23 Thread Avi Gross via Python-list
Bob, As others have noted, you have not made it clear how what you are doing is running "in parallel." I have a similar need where I have thousands of folders and need to do an analysis based on the contents of one at a time and have 8 cores available but the process may run for months if run lin

Re: PEP 594 cgi & cgitb removal

2019-05-23 Thread Gunnar Þór Magnússon
> nginx is the current hotness. CGI has not been hotness since the mid 90s. Serverless is the new hotness, and serverless is CGI. Technology is cyclical. -- https://mail.python.org/mailman/listinfo/python-list

RE: More CPUs doen't equal more speed

2019-05-23 Thread David Raymond
You really need to give more info on what you're doing in doit() to know what's going on. Are you using subprocess, threading, multiprocessing, etc? Going off of what you've put there those nested for loops are being run in the 1 main thread. If doit() kicks off a program and doesn't wait for it

Re: More CPUs doen't equal more speed

2019-05-23 Thread Chris Angelico
On Fri, May 24, 2019 at 5:37 AM Bob van der Poel wrote: > > I've got a short script that loops though a number of files and processes > them one at a time. I had a bit of time today and figured I'd rewrite the > script to process the files 4 at a time by using 4 different instances of > python. My

More CPUs doen't equal more speed

2019-05-23 Thread Bob van der Poel
I've got a short script that loops though a number of files and processes them one at a time. I had a bit of time today and figured I'd rewrite the script to process the files 4 at a time by using 4 different instances of python. My basic loop is: for i in range(0, len(filelist), CPU_COUNT): f

Re: Installation Problems with Python 3.7.3

2019-05-23 Thread Carolyn Evans
I got it working. Thanks On Mon, May 20, 2019 at 3:17 PM Igor Korot wrote: > Hi, > > On Mon, May 20, 2019 at 1:53 PM Carolyn Evans wrote: > > > > I am having trouble with re-installing python 3.7.3. > > Why do you need to reinstall? > What seems to be the problem? > > Thank you. > > > > > I ke

Re: How do you organize your virtual environments?

2019-05-23 Thread Chris Angelico
On Fri, May 24, 2019 at 12:42 AM Skip Montanaro wrote: > My way of thinking about virtual environments has always leaned in the > direction of a per-application setup, as that requires less > coordination (particularly when deploying to production), but I'm > willing to be convinced to move in few

How do you organize your virtual environments?

2019-05-23 Thread Skip Montanaro
Perhaps the subject isn't quite correct, but here's what I'm after. Suppose you have five applications, each going through a series of dev, test and prod phases. I will assume without further explanation or justification, that the dev phase is wholly within the purview of the developers who gets to

Handling an connection error with Twython

2019-05-23 Thread Cecil Westerhof
I am using Twython to post updates on Twitter. Lately there is now and then a problem with my internet connection. I am using: posted = twitter.update_status(status = message, in_reply_to_status_id = message_id, trim_user = T

Re: PEP 594 cgi & cgitb removal

2019-05-23 Thread Rhodri James
On 22/05/2019 19:29, Terry Reedy wrote: One of the factors being considered in removal decisions is the absence of anyone willing to list themselves in the expert's list https://devguide.python.org/experts/ as a maintainer for a module. At the moment, 3 other people have objected to the removal

Re: PEP 594 cgi & cgitb removal

2019-05-23 Thread Jon Ribbens via Python-list
On 2019-05-23, Paul Rubin wrote: > dieter writes: >> Should "cgi" disappear from the standard library > > It's also a concern that cgi may be disappearing from web servers. Last > I heard, nginx didn't support it. That's part of why I still use > apache, or (local only) even CGIHTTPServer.py.

EuroPython 2019: Monday and Tuesday activities for main conference attendees

2019-05-23 Thread M.-A. Lemburg
Although the main conference starts on Wednesday, July 10th, there’s already so much to do for attendees with the main conference ticket on Monday 8th and Tuesday 9th. Beginners’ Day and Sponsored Trainings -- You can come to the workshops and trainings venue a