[Mojolicious] Re: A pair of processes using 8% of CPU each
Hi Charly,
missing some details.
1.) OS Version
2.) Kernel Version
3.) mojo version.
4.) Do you use cpan(m) to install the lastest version or are you using OS
packages ?
I had similar problems but switched to a new mojo version and latest EV. (
cpanm EV).
For EV it should be kernel > 4.5
It seems that you started your app via daemon.
Try to use hypnotoad and check if there are changes. (got to the base dir
and enter hypnotoad -f script/yourscript).
Some hints about reading and writing to a process.
I use Mojo::IOLoop::ReadWriteFork togehter with the delay helper
($c->delay), which works perfectly.
Perhaps it's this your're looking for.
Rgds.
Hans
Am Donnerstag, 29. Juni 2017 16:10:46 UTC+2 schrieb Charlie Brady:
>
>
> On Thu, 29 Jun 2017, Jan Henning Thorsen wrote:
>
> > Hey,
> >
> > It doesn't look like "daemon" mode. It looks like you used "morbo" to
> start
> > the application.
>
> I don't know what makes you say that. I definitely was using "daemon" mode
> and was not using "morbo".
>
> I didn't track down exactly what code was making the difference in the
> poll timeout, but worked out that it was somewhere in the CGI plugin. I've
> eliminated the CGI plugin and now find that the poll timeout is back to
> 1000 ms, which a corresponding reduction in idle CPU.
>
> I don't know exactly where the fork() was which lead to the pair of
> processes. I did have some code with an open() reading from a pipe, but I
> wouldn't expect to see poll() in the child process in that case.
>
> Mystery not quite solved, but now that I am not using the CGI plugin I at
> least don't have an ongoing problem.
>
> Thanks for commenting. If you want to try to identify the problem with the
> CGI plugin I can re-insert that code.
>
> >
> >
> > On Wednesday, June 14, 2017 at 11:11:31 PM UTC+2, Charlie Brady wrote:
> > >
> > >
> > > I'm running a Mojolicious UI in daemon mode on a PowerPC embedded
> system
> > > (running Wind River Linux, perl 5.22.0, kernel 4.1.21) and notice
> > > surprisingly high CPU usage on a totally idle UI.
> > >
> > > I can see that there are two processes. Each process is in a polling
> loop
> > > with 10ms timeout. I don't expect to see two processes here, so I
> wonder
> > > whether this is normal or abnormal behaviour. If it is normal
> behaviour,
> > > can I tune down the 10ms timeout to something longer? I would like to
> see
> > > the process block until there is actual work to do.
> > >
> > > root@10:/service# ps fax | grep index.daemon-5
> > > 1418 pts/1S+ 0:00 \_ grep index.daemon
> > > 19303 ?Ssl 24:24 /etc/e-smith/web/functions/index.daemon.cgi
> > > 24397 ?Ssl 20:54 \_
> /etc/e-smith/web/functions/index.daemon.cgi
> > > root@10:/service# strace -p19303 -tt 2>&1 | head -5
> > > Process 19303 attached
> > > 17:05:08.095154 restart_syscall(<... resuming interrupted call ...>) =
> 0
> > > 17:05:08.104465 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) =
> 0
> > > (Timeout)
> > > 17:05:08.116478 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10)
> = 0
> > > (Timeout)
> > > 17:05:08.130120 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) =
> 0
> > > (Timeout)
> > > root@10:/service# strace -p24397 -tt 2>&1 | head -5
> > > Process 24397 attached
> > > 17:05:12.888643 restart_syscall(<... resuming interrupted call ...>) =
> 0
> > > 17:05:12.894814 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10)
> = 0
> > > (Timeout)
> > > 17:05:12.908082 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10)
> = 0
> > > (Timeout)
> > > 17:05:12.920771 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10)
> = 0
> > > (Timeout)
> > > root@10:/service# top | head -12
> > > top - 17:09:31 up 9 days, 7:59, 4 users, load average: 0.73, 0.59,
> 0.60
> > > Tasks: 98 total, 1 running, 95 sleeping, 0 stopped, 2 zombie
> > > %Cpu(s): 12.9 us, 5.7 sy, 3.8 ni, 75.7 id, 1.8 wa, 0.0 hi, 0.1
> si,
> > > 0.0 st
> > > KiB Mem : 995536 total, 5428 free, 317100 used, 673008
> > > buff/cache
> > > KiB Swap:65532 total,65448 free, 84 used. 594360 avail
> Mem
> > >
> > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+
> COMMAND
> > >
> > > 1440 root 20 03852 2280 1924 R 24.0 0.2 0:00.13 top
>
> > >
> > > 508 root 20 0 134908 69156 13896 S 12.0 6.9 465:50.05
> > > call_control
> > > 19303 root 20 0 87812 37720 4148 S 8.0 3.8 24:45.37
> > > /etc/e-smith/we
> > > 24397 root 20 0 87788 37184 3552 S 8.0 3.7 21:13.65
> > > /etc/e-smith/we
> > > 425 root 20 0 16068 10284 3640 S 4.0 1.0 33:32.22
> snmpd
> > >
> > >
> >
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop
[Mojolicious] Re: A pair of processes using 8% of CPU each
On Thu, 29 Jun 2017, Jan Henning Thorsen wrote:
> Hey,
>
> It doesn't look like "daemon" mode. It looks like you used "morbo" to start
> the application.
I don't know what makes you say that. I definitely was using "daemon" mode
and was not using "morbo".
I didn't track down exactly what code was making the difference in the
poll timeout, but worked out that it was somewhere in the CGI plugin. I've
eliminated the CGI plugin and now find that the poll timeout is back to
1000 ms, which a corresponding reduction in idle CPU.
I don't know exactly where the fork() was which lead to the pair of
processes. I did have some code with an open() reading from a pipe, but I
wouldn't expect to see poll() in the child process in that case.
Mystery not quite solved, but now that I am not using the CGI plugin I at
least don't have an ongoing problem.
Thanks for commenting. If you want to try to identify the problem with the
CGI plugin I can re-insert that code.
>
>
> On Wednesday, June 14, 2017 at 11:11:31 PM UTC+2, Charlie Brady wrote:
> >
> >
> > I'm running a Mojolicious UI in daemon mode on a PowerPC embedded system
> > (running Wind River Linux, perl 5.22.0, kernel 4.1.21) and notice
> > surprisingly high CPU usage on a totally idle UI.
> >
> > I can see that there are two processes. Each process is in a polling loop
> > with 10ms timeout. I don't expect to see two processes here, so I wonder
> > whether this is normal or abnormal behaviour. If it is normal behaviour,
> > can I tune down the 10ms timeout to something longer? I would like to see
> > the process block until there is actual work to do.
> >
> > root@10:/service# ps fax | grep index.daemon-5
> > 1418 pts/1S+ 0:00 \_ grep index.daemon
> > 19303 ?Ssl 24:24 /etc/e-smith/web/functions/index.daemon.cgi
> > 24397 ?Ssl 20:54 \_ /etc/e-smith/web/functions/index.daemon.cgi
> > root@10:/service# strace -p19303 -tt 2>&1 | head -5
> > Process 19303 attached
> > 17:05:08.095154 restart_syscall(<... resuming interrupted call ...>) = 0
> > 17:05:08.104465 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) = 0
> > (Timeout)
> > 17:05:08.116478 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> > (Timeout)
> > 17:05:08.130120 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) = 0
> > (Timeout)
> > root@10:/service# strace -p24397 -tt 2>&1 | head -5
> > Process 24397 attached
> > 17:05:12.888643 restart_syscall(<... resuming interrupted call ...>) = 0
> > 17:05:12.894814 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> > (Timeout)
> > 17:05:12.908082 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> > (Timeout)
> > 17:05:12.920771 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> > (Timeout)
> > root@10:/service# top | head -12
> > top - 17:09:31 up 9 days, 7:59, 4 users, load average: 0.73, 0.59, 0.60
> > Tasks: 98 total, 1 running, 95 sleeping, 0 stopped, 2 zombie
> > %Cpu(s): 12.9 us, 5.7 sy, 3.8 ni, 75.7 id, 1.8 wa, 0.0 hi, 0.1 si,
> > 0.0 st
> > KiB Mem : 995536 total, 5428 free, 317100 used, 673008
> > buff/cache
> > KiB Swap:65532 total,65448 free, 84 used. 594360 avail Mem
> >
> > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND
> >
> > 1440 root 20 03852 2280 1924 R 24.0 0.2 0:00.13 top
> >
> > 508 root 20 0 134908 69156 13896 S 12.0 6.9 465:50.05
> > call_control
> > 19303 root 20 0 87812 37720 4148 S 8.0 3.8 24:45.37
> > /etc/e-smith/we
> > 24397 root 20 0 87788 37184 3552 S 8.0 3.7 21:13.65
> > /etc/e-smith/we
> > 425 root 20 0 16068 10284 3640 S 4.0 1.0 33:32.22 snmpd
> >
> >
>
[Mojolicious] Re: A pair of processes using 8% of CPU each
Hey,
It doesn't look like "daemon" mode. It looks like you used "morbo" to start
the application.
On Wednesday, June 14, 2017 at 11:11:31 PM UTC+2, Charlie Brady wrote:
>
>
> I'm running a Mojolicious UI in daemon mode on a PowerPC embedded system
> (running Wind River Linux, perl 5.22.0, kernel 4.1.21) and notice
> surprisingly high CPU usage on a totally idle UI.
>
> I can see that there are two processes. Each process is in a polling loop
> with 10ms timeout. I don't expect to see two processes here, so I wonder
> whether this is normal or abnormal behaviour. If it is normal behaviour,
> can I tune down the 10ms timeout to something longer? I would like to see
> the process block until there is actual work to do.
>
> root@10:/service# ps fax | grep index.daemon-5
> 1418 pts/1S+ 0:00 \_ grep index.daemon
> 19303 ?Ssl 24:24 /etc/e-smith/web/functions/index.daemon.cgi
> 24397 ?Ssl 20:54 \_ /etc/e-smith/web/functions/index.daemon.cgi
> root@10:/service# strace -p19303 -tt 2>&1 | head -5
> Process 19303 attached
> 17:05:08.095154 restart_syscall(<... resuming interrupted call ...>) = 0
> 17:05:08.104465 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) = 0
> (Timeout)
> 17:05:08.116478 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> (Timeout)
> 17:05:08.130120 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 9) = 0
> (Timeout)
> root@10:/service# strace -p24397 -tt 2>&1 | head -5
> Process 24397 attached
> 17:05:12.888643 restart_syscall(<... resuming interrupted call ...>) = 0
> 17:05:12.894814 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> (Timeout)
> 17:05:12.908082 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> (Timeout)
> 17:05:12.920771 poll([{fd=21, events=POLLIN|POLLPRI|POLLOUT}], 1, 10) = 0
> (Timeout)
> root@10:/service# top | head -12
> top - 17:09:31 up 9 days, 7:59, 4 users, load average: 0.73, 0.59, 0.60
> Tasks: 98 total, 1 running, 95 sleeping, 0 stopped, 2 zombie
> %Cpu(s): 12.9 us, 5.7 sy, 3.8 ni, 75.7 id, 1.8 wa, 0.0 hi, 0.1 si,
> 0.0 st
> KiB Mem : 995536 total, 5428 free, 317100 used, 673008
> buff/cache
> KiB Swap:65532 total,65448 free, 84 used. 594360 avail Mem
>
> PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND
>
> 1440 root 20 03852 2280 1924 R 24.0 0.2 0:00.13 top
>
> 508 root 20 0 134908 69156 13896 S 12.0 6.9 465:50.05
> call_control
> 19303 root 20 0 87812 37720 4148 S 8.0 3.8 24:45.37
> /etc/e-smith/we
> 24397 root 20 0 87788 37184 3552 S 8.0 3.7 21:13.65
> /etc/e-smith/we
> 425 root 20 0 16068 10284 3640 S 4.0 1.0 33:32.22 snmpd
>
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Static cache header plugin
Hey,
I think you
want
https://metacpan.org/source/LDIDRY/Mojolicious-Plugin-StaticCache-0.01/lib/Mojolicious/Plugin/StaticCache.pm#L15
to be this:
if ($mode ne 'development' || $even_in_dev) {
And I think I would store $mode and $even_in_dev outside of the
"after_static" hook.
On Thursday, June 22, 2017 at 10:58:03 AM UTC+2, Luc Didry wrote:
>
> Hello,
>
> As discussed on the "Wishlist thread (fourth edition)"[1], I made a
> Mojolicious
> Plugin which add Cache-Control header on served static files in production
> (the
> header is not added in development mode):
> https://metacpan.org/pod/Mojolicious::Plugin::StaticCache
>
> It's basic and has very few options but it does the job.
>
> The code is on https://framagit.org/luc/mojolicious-plugin-staticcache
> and
> issues should be opened on
> https://framagit.org/luc/mojolicious-plugin-staticcache/issues.
>
> (if you don't have a framagit.org account, you can use Github
> authentication to
> get one in a few clicks).
>
> Comments and code contribution are welcome
>
> [1]
> https://groups.google.com/forum/#!searchin/mojolicious/wishlist|sort:relevance/mojolicious/JLZlJwkm0Ho/mFE8CFIcAAAJ
>
> --
> Luc
> https://fiat-tux.fr/
> https://luc.frama.io/
> Internet n'est pas compliqué, Internet est ce que vous en faites.
>
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Minion Locking problem (deadlock) !
I'm closing this thread since it has become a mess with those deleted messages. Don't do that. -- sebastian -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Minion Locking problem (deadlock) !
Hello Sebastian,
thanks.
i already corrected it and moved it to the worker process.
Also i solved my problem by using the retry_job if the lock is active which
works.
Rgds.
Hans.
Am Donnerstag, 29. Juni 2017 09:42:02 UTC+2 schrieb sri:
>
> > $app->minion->add_task(demo_job => sub {
> > my $job = shift;
> > my $retval = {};
> >
> > sleep 1 until $app->minion->lock('demo_job', 7200);
> >
> > $job->on(finished => sub {
> > my ($job, $result) = @_;
> > my $debugger = $app->debug_logger;
> > $debugger->debug(sprintf("Job %d with task %s finished\n",
> > $job->id, $job->task ));
> > $app->minion->unlock('demo_job');
> > });
> >
> > $job->on(failed => sub {
> > my ($job, $err) = @_;
> > my $debugger = $app->debug_logger;
> > $debugger->debug(sprintf("Job %d with task %s failed %s\n",
> > $job->id, $job->task, $err ));
> > $app->minion->unlock('demo_job');
> > });
> >
> > $retval->{data}->[0]->{demo_job} = int(rand(1000));
> > $retval->{success} = Cpanel::JSON::XS::true();
> > return $job->finish($retval);
> > });
>
> You can't use the finished and failed events from inside the task.
>
> --
> Sebastian Riedel
> http://mojolicio.us
> http://github.com/kraih
> http://twitter.com/kraih
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Minion Locking problem (deadlock) !
> $app->minion->add_task(demo_job => sub {
> my $job = shift;
> my $retval = {};
>
> sleep 1 until $app->minion->lock('demo_job', 7200);
>
> $job->on(finished => sub {
> my ($job, $result) = @_;
> my $debugger = $app->debug_logger;
> $debugger->debug(sprintf("Job %d with task %s finished\n",
> $job->id, $job->task ));
> $app->minion->unlock('demo_job');
> });
>
> $job->on(failed => sub {
> my ($job, $err) = @_;
> my $debugger = $app->debug_logger;
> $debugger->debug(sprintf("Job %d with task %s failed %s\n",
> $job->id, $job->task, $err ));
> $app->minion->unlock('demo_job');
> });
>
> $retval->{data}->[0]->{demo_job} = int(rand(1000));
> $retval->{success} = Cpanel::JSON::XS::true();
> return $job->finish($retval);
> });
You can't use the finished and failed events from inside the task.
--
Sebastian Riedel
http://mojolicio.us
http://github.com/kraih
http://twitter.com/kraih
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Minion Locking problem (deadlock) !
Hi,
i highly appreciate your work for minion and the latest version which
supports locks.
i really need something like queuing by using the lock mech. Before i used
file locks via flock.
I have a system which only can process one job by another.
The problem i have is, that $minio->lock return a deadlock situation.
A simple demo job:
$app->minion->add_task(demo_job => sub {
my $job = shift;
my $retval = {};
sleep 1 until $app->minion->lock('demo_job', 7200);
$job->on(finished => sub {
my ($job, $result) = @_;
my $debugger = $app->debug_logger;
$debugger->debug(sprintf("Job %d with task %s finished\n", $job->
id, $job->task ));
$app->minion->unlock('demo_job');
});
$job->on(failed => sub {
my ($job, $err) = @_;
my $debugger = $app->debug_logger;
$debugger->debug(sprintf("Job %d with task %s failed %s\n", $job->
id, $job->task, $err ));
$app->minion->unlock('demo_job');
});
$retval->{data}->[0]->{demo_job} = int(rand(1000));
$retval->{success} = Cpanel::JSON::XS::true();
return $job->finish($retval);
});
if i enqueue e.g. 8 jobs, some of them fail with:
"result" : "DBD::Pg::st execute failed: ERROR: deadlock detected\nDETAIL:
Process 43250 waits for ExclusiveLock on relation 19753 of database 19627;
blocked by process 43294.\nProcess 43294 waits for ExclusiveLock on
relation 19753 of database 19627; blocked by process 43250.\nHINT: See
server log for query details.\nCONTEXT: SQL statement \"lock table
minion_locks in exclusive mode\"\nPL/pgSQL function
minion_lock(text,integer,integer) line 6 at SQL statement at
/opt/perl5/perls/perl-5.24.1t/lib/site_perl/5.24.1/Minion/Backend/Pg.pm
line 90.\n"
I thought when using lock and it's locked than i can wait until the lock is
free ?
If that's not possible i will return to my working flock mech.
Perhaps anyone can point me in the right direction.
Kind regards.
Hans
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
