Re: Parallel(?) programming with python

I would like to thank everybody who answered my question. The insight was very informative. This seems to be one of the few newsgroups still alive and kicking, with a lot of knowledgeable people taking the time to help others. I like how quick and easy it is to post questions and receive

Re: Parallel(?) programming with python

Dennis Lee Bieber wrote at 2022-8-10 14:19 -0400: >On Wed, 10 Aug 2022 19:33:04 +0200, "Dieter Maurer" > ... >>You could also use the `sched` module from Python's library. > >Time to really read the library reference manual again... > > Though if I read this correctly, a long

Re: Parallel(?) programming with python

Please let me know if that is okay. On Wed, Aug 10, 2022 at 7:46 PM <2qdxy4rzwzuui...@potatochowder.com> wrote: > On 2022-08-09 at 17:04:51 +, > "Schachner, Joseph (US)" wrote: > > > Why would this application *require* parallel programming? This could > > be done in one, single thread

Re: Parallel(?) programming with python

Thanks again for the info. On Wed, Aug 10, 2022 at 9:31 PM Peter J. Holzer wrote: > On 2022-08-10 14:19:37 -0400, Dennis Lee Bieber wrote: > > On Wed, 10 Aug 2022 19:33:04 +0200, "Dieter Maurer" > > > declaimed the following: > > >Schachner, Joseph (US) wrote at 2022-8-9 17:04 +: > > >>Why

Re: Parallel(?) programming with python

On 2022-08-10 14:19:37 -0400, Dennis Lee Bieber wrote: > On Wed, 10 Aug 2022 19:33:04 +0200, "Dieter Maurer" > declaimed the following: > >Schachner, Joseph (US) wrote at 2022-8-9 17:04 +: > >>Why would this application *require* parallel programming? This > >>could be done in one, single

RE: Parallel(?) programming with python

Of Dieter Maurer Sent: Wednesday, August 10, 2022 1:33 PM To: Schachner, Joseph (US) Cc: Andreas Croci ; python-list@python.org Subject: RE: Parallel(?) programming with python Schachner, Joseph (US) wrote at 2022-8-9 17:04 +: >Why would this application *require* parallel programm

Re: Parallel(?) programming with python

On 2022-08-09 at 17:04:51 +, "Schachner, Joseph (US)" wrote: > Why would this application *require* parallel programming? This could > be done in one, single thread program. Call time to get time and save > it as start_time. Keep a count of the number of 6 hour intervals, > initialize it

Re: Parallel(?) programming with python

On Wed, 10 Aug 2022 19:33:04 +0200, "Dieter Maurer" declaimed the following: >Schachner, Joseph (US) wrote at 2022-8-9 17:04 +: >>Why would this application *require* parallel programming? This could be >>done in one, single thread program. Call time to get time and save it as

RE: Parallel(?) programming with python

Schachner, Joseph (US) wrote at 2022-8-9 17:04 +: >Why would this application *require* parallel programming? This could be >done in one, single thread program. Call time to get time and save it as >start_time. Keep a count of the number of 6 hour intervals, initialize it to >0. You

RE: Parallel(?) programming with python

, August 8, 2022 6:47 AM To: python-list@python.org Subject: Parallel(?) programming with python tI would like to write a program, that reads from the network a fixed amount of bytes and appends them to a list. This should happen once a second. Another part of the program should take the list

Re: Parallel(?) programming with python

On Mon, 8 Aug 2022 19:39:27 +0200, Andreas Croci declaimed the following: > >Do you mean queues in the sense of deque (the data structure)? I ask >because I can see the advantage there when I try to pop data from the >front of it, but I don't see the sense of the following statement ("than

Re: Parallel(?) programming with python

Queues are better than lists for concurrency. If you get the right kind, they have implicit locking, making your code simpler and more robust at the same time. CPython threading is mediocre for software systems that have one or more CPU-bound threads, and your FFT might be CPU-bound. Rather

Re: Parallel(?) programming with python

On 09Aug2022 00:22, Oscar Benjamin wrote: >On Mon, 8 Aug 2022 at 19:01, Andreas Croci wrote: >> Basically the question boils down to wether it is possible to have >> parts >> of a program (could be functions) that keep doing their job while other >> parts do something else on the same data, and

RE: Parallel(?) programming with python

: Parallel(?) programming with python Andreas Croci writes: >Basically the question boils down to wether it is possible to have >parts of a program (could be functions) that keep doing their job while >other parts do something else on the same data, and what is the best >way to do

Re: Parallel(?) programming with python

On Mon, 8 Aug 2022 at 19:01, Andreas Croci wrote: > > tI would like to write a program, that reads from the network a fixed > amount of bytes and appends them to a list. This should happen once a > second. > > Another part of the program should take the list, as it has been filled > so far, every

Re: Parallel(?) programming with python

On 08Aug2022 11:20, Stefan Ram wrote: >Andreas Croci writes: >>Basically the question boils down to wether it is possible to have parts >>of a program (could be functions) that keep doing their job while other >>parts do something else on the same data, and what is the best way to do >>this. > >

Re: Parallel(?) programming with python

On 2022-08-08 13:53:20 +0200, Andreas Croci wrote: > I'm in principle ok with locks, if it must be. What I fear is that the lock > could last long and prevent the function that writes into the list from > doing so every second. With an FFT on a list that contains a few bytes taken > every second

Re: Parallel(?) programming with python

> On 8 Aug 2022, at 20:24, MRAB wrote: > > On 2022-08-08 12:20, Stefan Ram wrote: >> Andreas Croci writes: >>> Basically the question boils down to wether it is possible to have parts of >>> a program (could be functions) that keep doing their job while other parts >>> do something else on

Re: Parallel(?) programming with python

On 2022-08-08 12:20, Stefan Ram wrote: Andreas Croci writes: Basically the question boils down to wether it is possible to have parts of a program (could be functions) that keep doing their job while other parts do something else on the same data, and what is the best way to do this.

Re: Parallel(?) programming with python

On 8/8/2022 4:47 AM, Andreas Croci wrote: tI would like to write a program, that reads from the network a fixed amount of bytes and appends them to a list. This should happen once a second. Another part of the program should take the list, as it has been filled so far, every 6 hours or so,

Re: Parallel(?) programming with python

On Mon, 8 Aug 2022 12:47:26 +0200, Andreas Croci declaimed the following: >tI would like to write a program, that reads from the network a fixed >amount of bytes and appends them to a list. This should happen once a >second. > Ignoring leap seconds, there are 86400 seconds in a day --

RE: Parallel(?) programming with python

>> But, an easier and often >> better option for concurrent data access is use a (relational) >> database, then the appropriate transaction isolation levels >> when reading and/or writing. >> > > That would obviusly save some coding (but would introduce the need to > code the interaction with the

Re: Parallel(?) programming with python

Thank you for your reply. On 08.08.22 14:55, Julio Di Egidio wrote: Concurrent programming is quite difficult, plus you better think in terms of queues than shared data... Do you mean queues in the sense of deque (the data structure)? I ask because I can see the advantage there when I try

Re: Parallel(?) programming with python

Thanks for your reply. On 08.08.22 13:20, Stefan Ram wrote: Yes, but this is difficult. If you ask this question here, you might not be ready for this. Indeed. I haven't learned it yet myself, but nevertheless tried to write a small example program quickly, which might still

Parallel(?) programming with python

tI would like to write a program, that reads from the network a fixed amount of bytes and appends them to a list. This should happen once a second. Another part of the program should take the list, as it has been filled so far, every 6 hours or so, and do some computations on the data (a

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

On Thu, Jun 16, 2016 at 2:45 AM, meInvent bbird wrote: > the name in c# is not called concurrent list, it is called > blockingcollection > > dictionary called concurrent dictionary > > thread safe these kind of things > >

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

how can list be synchronized when multiprocessor working in it? will one thread updating non-updated version, but another processor updating the version? On Thursday, June 16, 2016 at 4:30:33 PM UTC+8, Steven D'Aprano wrote: > On Thursday 16 June 2016 17:28, meInvent bbird wrote: > > > is

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

the name in c# is not called concurrent list, it is called blockingcollection dictionary called concurrent dictionary thread safe these kind of things https://msdn.microsoft.com/en-us/library/dd267312(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/dd997369(v=vs.110).aspx

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

On Thursday 16 June 2016 17:28, meInvent bbird wrote: > is there like c# have concurrent list ? What is a concurrent list? Can you link to the C# documentation for this? To me, "concurrent" describes a style of execution flow, and "list" describes a data structure. I am struggling to

is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

is there like c# have concurrent list ? i find something these, but how can it pass an initlist list variable is it doing the same function as itertools.combinations ? def comb(n, initlist): # the argument n is the number of items to select res = list(itertools.combinations(initlist,

Re: parallel programming in Python

Hehe, I just asked this question a few days ago but I didn't become much cleverer: http://www.gossamer-threads.com/lists/python/python/985701 Best, Laszlo On Thu, May 10, 2012 at 2:14 PM, Jabba Laci jabba.l...@gmail.com wrote: Hi, I would like to do some parallel programming with Python

Re: parallel programming in Python

: Hehe, I just asked this question a few days ago but I didn't become much cleverer: http://www.gossamer-threads.com/lists/python/python/985701 Best, Laszlo On Thu, May 10, 2012 at 2:14 PM, Jabba Lacijabba.l...@gmail.com wrote: Hi, I would like to do some parallel programming with Python but I

parallel programming in Python

Hi, I would like to do some parallel programming with Python but I don't know how to start. There are several ways to go but I don't know what the differences are between them: threads, multiprocessing, gevent, etc. I want to use a single machine with several cores. I want to solve problems like

Re: parallel programming in Python

On 05/10/2012 08:14 AM, Jabba Laci wrote: Hi, I would like to do some parallel programming with Python but I don't know how to start. There are several ways to go but I don't know what the differences are between them: threads, multiprocessing, gevent, etc. I want to use a single machine

Re: parallel programming in Python

On Thu, May 10, 2012 at 8:14 AM, Jabba Laci jabba.l...@gmail.com wrote: What's the best way? From what I've heard, http://scrapy.org/ . It is a single-thread single-process web crawler that nonetheless can download things concurrently. Doing what you want in Scrapy would probably involve

Re: parallel programming in Python

...@davea.name wrote: On 05/10/2012 08:14 AM, Jabba Laci wrote: Hi, I would like to do some parallel programming with Python but I don't know how to start. There are several ways to go but I don't know what the differences are between them: threads, multiprocessing, gevent, etc. I want to use

Re: parallel programming in Python

On 05/10/2012 06:46 AM, Devin Jeanpierre wrote: On Thu, May 10, 2012 at 8:14 AM, Jabba Laci jabba.l...@gmail.com wrote: What's the best way? From what I've heard, http://scrapy.org/ . It is a single-thread single-process web crawler that nonetheless can download things concurrently. Yes,