Re: [PHP-DEV] Gearman support for php-fpm?
Rasmus, I can work on it if no one has it already. Warm Regards, Patrick On Thu, Jun 10, 2010 at 7:56 AM, Rasmus Lerdorf ras...@lerdorf.com wrote: Has anybody thought about adding Gearman support to the fpm sapi? Managing Gearman workers is pretty much identical to managing fastcgi workers, so it doesn't seem like much of a stretch. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
The big difference is that different processes do different jobs in Gearman. All PHP workers in fpm just run what ever code is handed to them. How do you handle that? Brian. http://brian.moonspot.net/ On 6/10/10 9:56 AM, Rasmus Lerdorf wrote: Has anybody thought about adding Gearman support to the fpm sapi? Managing Gearman workers is pretty much identical to managing fastcgi workers, so it doesn't seem like much of a stretch. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
On 06/10/2010 17:20, Brian Moon wrote: The big difference is that different processes do different jobs in Gearman. All PHP workers in fpm just run what ever code is handed to them. How do you handle that? Threading ;) ? On 6/10/10 9:56 AM, Rasmus Lerdorf wrote: Has anybody thought about adding Gearman support to the fpm sapi? Managing Gearman workers is pretty much identical to managing fastcgi workers, so it doesn't seem like much of a stretch. Best regards, Fred -- Frédéric Hardy : Architecte d'application/Admin. système/Ergonome Status : En recherche d'emploi CV : http://blog.mageekbox.net/public/cv.frederic.hardy.pdf Blog : http://blog.mageekbox.net Twitter : http://twitter.com/mageekguy -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
On 6/10/10 8:30 AM, Frederic Hardy wrote: On 06/10/2010 17:20, Brian Moon wrote: The big difference is that different processes do different jobs in Gearman. All PHP workers in fpm just run what ever code is handed to them. How do you handle that? Threading ;) ? Definitely not. Gearman is actually the answer to a lot of problems where people think they need threads. As far as different workers doing different jobs, I don't see the issue here. fpm doesn't need to know the details. All it has to know is that it needs to maintain pools of php processes. Each pool would correspond to a Gearman job. The workers can take care of registering themselves with gearmand. All fpm has to do is start up and maintain N processes per worker pool. [gearman] worker1.path = /path/to/worker1.php worker1.pm = dynamic worker1.pm.max_children = 10 worker1.pm.start_servers = 5 worker2.path = /path/to/worker2.php worker2.pm = dynamic worker2.pm.max_children = 4 worker2.pm.start_servers = 2 ... or however you do multiple nested conf blocks in the fpm .ini file. In fact, this may not need to be gearman-specific. We could just call it cli job management or something like that if we left it completely up to the workers to register with the gearmand. Alternatively, we could make fpm Gearman-aware and register the workers from there, but I am not sure I see much of an advantage to that other than making the worker code slightly simpler. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
All fpm workers would be called using the same function name (e.g. runphp). This allows you to reuse a gearmand instance for other workers too. The advantages of fpm vs gearman would be mostly performance (it uses ping and requires an underlying layer on top of tcp/ip), followed by dynamic spawning and crash recovery. Think there are more but I can't access the website from over here ;) On 6/10/10, Brian Moon br...@moonspot.net wrote: The big difference is that different processes do different jobs in Gearman. All PHP workers in fpm just run what ever code is handed to them. How do you handle that? Brian. http://brian.moonspot.net/ On 6/10/10 9:56 AM, Rasmus Lerdorf wrote: Has anybody thought about adding Gearman support to the fpm sapi? Managing Gearman workers is pretty much identical to managing fastcgi workers, so it doesn't seem like much of a stretch. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- -- Tjerk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
On 6/10/10 8:53 AM, Tjerk Anne Meesters wrote: All fpm workers would be called using the same function name (e.g. runphp). This allows you to reuse a gearmand instance for other workers too. I don't think we would need to require that. Obviously you could build your Gearman worker such that it registers runphp and has a big switch in it to run various types of jobs and fpm would just need to manage a single pool of workers. But, assuming fpm could manage multiple pools, I don't see the problem with having different types of workers. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
[gearman] worker1.path = /path/to/worker1.php worker1.pm = dynamic worker1.pm.max_children = 10 worker1.pm.start_servers = 5 worker2.path = /path/to/worker2.php worker2.pm = dynamic worker2.pm.max_children = 4 worker2.pm.start_servers = 2 So, this is a proposed new entry in the ini format? I am a bit ignorant about fpm, I admit. This would require a good bit of fiddling for our gearman set up to work with it. We run X workers that know how to do all jobs available and then 1 + n (defined by ini file) that only do one job for each available job. This gives us good coverage with controlled process load. I guess it would just require a lot more configuration than we have now. For example, could we pass parameters to the script? Having a separate script plus a separate class for every function quickly makes a lot of files. I will simply stay tuned and see where it goes. FWIW, this is my GearmanManager that basically does this already via PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not dynamically ramp up more workers like FPM can. But, I would never run it that way anyhow. I like my processes under control. Brian. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Gearman support for php-fpm?
On 6/10/10 9:31 AM, Brian Moon wrote: [gearman] worker1.path = /path/to/worker1.php worker1.pm = dynamic worker1.pm.max_children = 10 worker1.pm.start_servers = 5 worker2.path = /path/to/worker2.php worker2.pm = dynamic worker2.pm.max_children = 4 worker2.pm.start_servers = 2 So, this is a proposed new entry in the ini format? I am a bit ignorant about fpm, I admit. This would require a good bit of fiddling for our gearman set up to work with it. We run X workers that know how to do all jobs available and then 1 + n (defined by ini file) that only do one job for each available job. This gives us good coverage with controlled process load. I guess it would just require a lot more configuration than we have now. For example, could we pass parameters to the script? Having a separate script plus a separate class for every function quickly makes a lot of files. I will simply stay tuned and see where it goes. FWIW, this is my GearmanManager that basically does this already via PHP+pcntl. http://github.com/brianlmoon/GearmanManager. It can not dynamically ramp up more workers like FPM can. But, I would never run it that way anyhow. I like my processes under control. Yes, I know, I have seen it. Process management and signal handling like that really isn't something that should be done in PHP as far as I am concerned. And FPM already does all of this and more. It just needs a slight tweak to be able to manage any sort of php process. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
