[Mojolicious] Re: curl works, Mojo::UserAgent doesn't
It's possible the `-k` option to curl is the difference. The equivalent in
Mojo::UserAgent is `$ua->insecure(1)` (which you can chain off of).
Scott
On Wednesday, May 13, 2020 at 3:56:26 PM UTC-6, Michael Lackhoff wrote:
>
> I had quite a bit of success recently using Mojo::UserAgent so I tried to
> replace a curl command to do a backup of my Fritz.box router with M::U.
>
> Here is the curl command:
> curl -s -k -o $OUT --form sid=$SID --form ImportExportPassword=$BAKPWD \
> --form ConfigExport= http://fritz.box/cgi-bin/firmwarecfg
>
> It should be equivalent to this M::U request:
>
> my $tx = $ua->build_tx(
> POST => 'http://fritz.box/cgi-bin/firmwarecfg' =>
> {
> 'Accept' => '*/*',
> 'Content-Type' => 'multipart/form-data',
> } => form => {
> sid => $SID,
> ImportExportPassword => $BAKPWD,
> ConfigExport => '',
> }
> );
>
> # for debugging:
> print $tx->req->to_string;
>
> $tx = $ua->start($tx);
> $tx->res->save_to($OUT);
>
> As far as I can tell both the headers and the POST body is very much the
> same (except the boundary value to separate the form fields) but to my
> surprise the curl command works ($OUT is the backup file) but with the M::U
> version $OUT consists of some HTML output indicating an error.
>
> If I could see a difference I could try to better adjust my script but as
> I said, they look very much the same (I compared it with the -v and
> --trace-ascii output of curl), so I run out of ideas what could trigger the
> differnt response of my Fritz.box.
> Any ideas? At the moment I just solve it by using the curl command with
> "system" but I would prefer a Perl-only solution and what is even more
> important to me: I want to understand what is going on here.
>
> -Michael
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mojolicious/61ee3b8a-ca87-424d-91e1-f807001afa37%40googlegroups.com.
[Mojolicious] Re: Testing internal method
Test::Mojo can help you test your views and controllers, as well as your
internal helpers:
https://metacpan.org/pod/distribution/Mojolicious/lib/Mojolicious/Guides/Testing.pod#Testing-application-helpers
If you are wanting to test your application *model* (i.e., non-Mojolicious
code), you should probably use standard Test::More to create an object,
exercise its methods, etc.
Scott
On Monday, April 27, 2020 at 4:55:43 PM UTC-6, Alberto Mijares wrote:
>
> Hi guys,
>
> I've been reading about testing and can't figure out how to test
> internal methods; meaning those methods not directly associated to a
> route.
>
> I'm sure most of you know what I'm talking about but just for the
> record. Let's say you have in App::Controller::This
>
> sub list {
> my $self = shift;
> $self->render(text => $self->filter($value));
> }
>
> sub filter {
> my $self = shift;
> for @list {push (@final, $line) if ($line =~ /something/);}
> return \@final
> }
>
> And in lib/App.pm you have
>
> $c->get('/list)->to('this#list');
>
> So, I want to test returned values from App::Controller::This->filter.
>
> Thanks in advance for your help.
>
>
> Alberto Mijares
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mojolicious/6fd72956-78ed-4922-bf30-e3cad431e5e0%40googlegroups.com.
Re: [Mojolicious] Query on read timeout in Mojo/IOLoop/Stream.pm
Hi Ganesh, Maybe this thread will give you some ideas: https://groups.google.com/d/msg/mojolicious/y43kqu06sgg/jLvSkSYvAgAJ Scott On Sunday, September 30, 2018 at 10:19:45 PM UTC-6, Ganesh Udupa wrote: > > Can someone help here? > Hi Sebastian would you know about th > > On Fri, Sep 28, 2018 at 12:20 PM Ganesh Udupa > wrote: > >> >> https://github.com/mojolicious/mojo/blob/master/lib/Mojo/IOLoop/Stream.pm#L40 >> Can someone help me understand what exactly is the timeout here?. I >> understand that we use Non blocking I/O and it might retry. >> I wanted to understand the behavior when it fails to read in 15 seconds. >> Will the method _read ( >> https://github.com/mojolicious/mojo/blob/master/lib/Mojo/IOLoop/Stream.pm#L98 >> ) >> throw an error? . Is there a way to increase this timeout from 15 seconds? >> The reason I ask is that, in our application, read_ is ending up with an >> error randomly. ( Very rare). Since this error is not printed, we are >> unable to root cause it. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Mojolicious" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mojolicious...@googlegroups.com . >> To post to this group, send email to mojol...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/mojolicious. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Configuration override via Test::Mojo not being used
For anyone looking for a working example of mocking a service in a test
file and overriding the configuration (standard style disclaimers apply),
here is the equivalent of the original (non-working) post using a full
Mojolicious app:
use Mojo::Base -strict;
use Test::More;
use Test::Mojo;
package Foo;
use Mojo::Base 'Mojolicious';
sub startup {
my $app = shift;
$app->plugin(Config => {default => {jack => 'squat'}});
$app->routes->get('/' => sub {
my $c = shift;
$c->render(text => $c->app->config->{jack});
});
}
package main;
my $t = Test::Mojo->new(Foo => {jack => 'flack'});
$t->get_ok('/')
->content_is('flack')
->or(sub { say STDERR $t->app->dumper(shift->tx->res->body) });
done_testing();
In my case, I have a second lite app I've loaded in the test file that I
can now point to use the mocked instance—handy to have one test file stand
up and tear down the mocked service along with the tests.
Scott
On Thursday, September 27, 2018 at 10:01:16 AM UTC-6, sri wrote:
>
> > Config overriding works only for full apps
>
> Actually that's not true, but i see this here is a special case.
> Config overrides only
> work with full apps and instances of lite apps. Lite apps in classes
> are a pretty
> bad hack and do not work here.
>
> --
> Sebastian Riedel
> https://mojolicious.org
> https://github.com/kraih
> https://twitter.com/kraih
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Configuration override via Test::Mojo not being used
I should have added that I’m running 8.01. -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Configuration override via Test::Mojo not being used
I have a package Foo that implements a Mojolicious::Lite application. It
has some default configuration. I instantiate this app via Test::Mojo and
attempt to override the configuration, but the old (default) value is
always used instead:
use Mojo::Base -strict;
use Test::More;
use Test::Mojo;
package Foo;
use Mojolicious::Lite -signatures;
plugin Config => {default => {jack => 'squat'}};
get '/' => sub ($c) { $c->render(text => $c->app->config->{jack}) };
package main;
my $t = Test::Mojo->new(Foo => {jack => 'flack'});
$t->get_ok('/')
->content_is('flack')
->or(sub { say STDERR $t->app->dumper(shift->tx->res->body) });
done_testing();
I've tried this same technique putting the Foo class in a separate file,
putting the configuration into a file (i.e., not using the `default`
attribute for Config), etc. and it always comes down to the override
configuration isn't being used by the app. Anyone have any ideas?
Scott
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Return mojo::pg result from the model to the controller to be rendered
I'll expand sri's reply a little. In your model:
sub save {
...
return $self->pg->db->update_p(...)
}
and in your controller:
sub my_route {
my $c = shift;
...
$c->model->save(...) ## save now returns a promise
->then(sub { ... render success here })
->catch(sub { ... render failure here });
}
This keeps the concerns of rendering out of your model.
Scott
On Monday, September 24, 2018 at 5:00:41 AM UTC-6, Nacho B wrote:
>
> Hi!
>
> As some of my controllers are getting too fat, I am relaying some things
> to models, starting with the database queries.
>
> Even when saving data I need to return something, an error or a true. But
> I don't want to render from the callback in the model. I would like to
> return the output in the callback to the controller, but I don't know how.
>
> I am using the controller-model structure from the blog example (
> https://github.com/mojolicious/mojo-pg/tree/master/examples/blog). It's
> very clean and easy to understand, but I don't know if the non-blocking way
> can fit in it…
>
>
>
> // in the 'party' model
>
> sub save {
> my ($self, $id, $party) = @_;
> $self->pg->db->update('party_t', $party, {id => $id} => sub {
> my ($db, $err, $results) = @_;
> if $err {
> // ... must I render here?? as JSON with the error code and text
> }
> else {
> // ... must I render here?? as JSON with the OK
> }
> });
>
> // I would like to return something to the controller!! from here, or
> from the callback
> }
>
>
>
> And for the controller… if I use $self->render_later… do the controller
> know that the render will occur in the model?
>
> Any suggestions? All the non-blocking tutorials and docs use just a "say"
> or a render, but it seems that I want to "break" the callback. Maybe I need
> another structure.
>
> Thank you in advance!
> Nacho B.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Mojo::JWT - error handling
Mojo::JWT::decode can die, so you'll want to wrap that in an eval() and
trap the exception.
Scott
On Friday, May 18, 2018 at 1:57:11 PM UTC-6, Luc Larochelle wrote:
>
> I'm using Mojo::JWT to produde a token and verify authorization on routes
> (using the given token, with the same secret)
>
> Thing is , when the token is invalid, an error is being thrown and the
> application crashes. I'd like to contain the error and deal with it.
>
> What am I missing ? Here's the sample code
>
>
>
> use Mojo::JWT;
> use Mojolicious::Lite;
> use Data::Dumper;
>
> helper check_token => sub {
>
> my $self = shift;
> my $token = $self->req->headers->authorization;
> my $jwt = Mojo::JWT->new(secret => 'mytest');
> return $jwt->decode($token);
>
> };
>
> any '/auth' => sub {
> my $c = shift;
> my $claims = $c->check_token;
>
> if (!($claims)) {
> $c->render(json => {status => "error", data => { message
> => "Please authenticate" }});
> } else {
> $c->render(json => $claims);
> }
>
> };
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Handling connection timeout cleanly when using Mojo::UserAgent to perform an HTTP request
Hi Jeremy,
It looks like the event loop timeout (MOJO_INACTIVITY_TIMEOUT) is
triggering before your connection timeout. Try setting that environment
value longer than the user agent timeouts (or set your user agent connect
and request timeouts shorter) and see if the error in the logs goes away.
Your "else" block should trigger with the timeout error in there.
I had a similar experience recently:
https://groups.google.com/forum/#!topic/mojolicious/y43kqu06sgg
Scott
On Wednesday, January 10, 2018 at 7:47:06 PM UTC-7, Jeremy Begg wrote:
>
> Hi,
>
> We have a Mojolicious web application which in general works quite nicely.
> Recently we had a need to add the capability to get some of the
> application data from a remote device.
> The remote device might be offline or slow so I wrote a Controller action
> to use Mojo::UA in non-blocking mode.
> It seems to work fine when the remote device is active, but it generates
> spurious error output when the remote device is offline.
> For example,
>
> [Wed Jan 10 18:36:31 2018] [debug] GET "/dashboard/hfdata/179"
> [Wed Jan 10 18:36:31 2018] [debug] Routing to controller
> "Frontend::Controller::Dashboard" and action "hf_proxy"
> [Wed Jan 10 18:36:31 2018] [debug] Initiating request for live data from
> http://(removed)
> [Wed Jan 10 18:36:31 2018] [debug] Request for live data failed;
> url=http://(removed)
> code=500 msg=Connect timeout
> Mojo::Reactor::Poll: Timer failed: Connection already closed at /usr/local
> /share/perl/5.22.1/Mojo/UserAgent.pm line 251.
>
> All but the last line are output by my Controller action.
> The last line is output by Mojolicious from its internals, writing
> directly to STDOUT (or STDERR, I suppose).
>
> Here is my Controller action (invoked via a Mojolicious route), is it
> doing something wrong?
> (I have obfuscated the remote device's URL in the example above and in the
> code below.)
>
> package Frontend::Controller::Dashboard;
>
> use Mojo::Base 'Mojolicious::Controller';
>
> sub hf_proxy {
>
> my $self = shift;
> my $user = $self->stash->{user};
> my $did = $self->param('did');
>
> my $device = $user->device_by_id($did);
> $self->stash(device => $device);
>
> if (!$device) {
> # Non-existent device ID, or not owned by this user
> $self->app->log->error("No device $did for $user");
> return $self->render(status => 404, json => { message => 'Invalid
> device ID for this user' } );
> }
>
> # Use the Mojolicious UA to query the myWatt board. See the article
> # "Blocking vs non-blocking 101" on the Mojolicious WiKi for
> explanation.
> my $ip = $device->ip;
> my $dhash = $device->device_id_hash;
> my $url = "http://(removed)";
> $self->app->log->debug("Initiating request for live data from $url");
> $self->render_later;
> $self->ua->get($url => sub {
> my ($ua, $tx) = (@_);
> if (my $res = $tx->success) {
> $self->app->log->debug("Got live data: ".$res->body);
> $self->render(data => $res->body, format => 'json');
> }
> else {
> my $error = $tx->error;
> my $code = $error->{code} || 500;
> my $msg = $error->{message} || 'Connection failed';
> $self->app->log->debug("Request for live data failed;
> url=$url code=$code msg=$msg");
> $self->render(status => $code, json => { message => $msg });
> }
> });
>
> }
>
>
>
> FYI, we are running on Ubuntu 16.4 LTS:
>
> jeremy@devtest:~$ mojo version
> CORE
> Perl(v5.22.1, linux)
> Mojolicious (7.44, Doughnut)
>
> OPTIONAL
> EV 4.0+ (n/a)
> IO::Socket::Socks 0.64+ (n/a)
> IO::Socket::SSL 1.94+ (2.024)
> Net::DNS::Native 0.15+ (n/a)
> Role::Tiny 2.01+(2.01)
>
>
> Thanks,
>
> Jeremy Begg
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Minion: better way to handle delayed jobs?
The jobs may take more than 1s to finish, but the backend web service I hit (which is out of my control) gets angry if things come in too fast. My worker is doing non-blocking HTTP requests, so while there may be several jobs in flight, I'm trying to keep the requests/second low. Originally I was going to just implement this with a Mojo::IOLoop timer, but if my service dies during a batch, I'd have to implement some durable job store... like Minion :) I'll try to work around this with some special logic in the worker or something. Scott On Friday, December 15, 2017 at 10:08:56 AM UTC-7, sri wrote: > > I'm adding many jobs into a queue (Pg backend) with something like this: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-inject >> >> Each job has a delay that increments by 1s. My worker handles the jobs: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-worker >> >> but the jobs come in bursts every 5 seconds, as you can see in the >> timestamps in the output: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-output-log >> >> Can anyone recommend a better technique for ensuring jobs don't run >> before their time—and run as close to their time as possible? Am I using >> 'delay' wrong? >> > > This is a tough case. Delays are definitely not meant for 1s accuracy, > there will always > be delays (haha...i'll see myself out...). I actually find it odd that you > have so many jobs > that take less than a second to finish, maybe a job queue is not the right > tool for the job? > Most of my background jobs are more in the 15s-15m range > > You could also use job dependencies, so later jobs only run when the > earlier ones are > finished. > > -- > sebastian > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Question about MOJO_INACTIVITY_TIMEOUT effects and Mojo::IOLoop::Stream
And then I find the answer in the FAQ:
In Mojolicious <http://www.mojolicious.org/perldoc/Mojolicious> applications
you can also use the helper "inactivity_timeout" in
Mojolicious::Plugin::DefaultHelpers
<http://www.mojolicious.org/perldoc/Mojolicious/Plugin/DefaultHelpers#inactivity_timeout>
to
change it on demand for each connection individually. This timeout always
applies, so you might have to tweak it for applications that take a long
time to process a request.
Not sure how I missed that. As you were.
Scott
On Tuesday, September 5, 2017 at 2:21:31 PM UTC-6, Scott Wiersdorf wrote:
>
> I have a slow HTTP service I'm writing a proxy for. If I hit the service
> directly using curl, it looks like this:
>
> $ time curl http://localhost:/?timer=10
> {"message":"waited 10 seconds"}
> real 0m10.018s
> user 0m0.005s
> sys 0m0.005s
>
>
> I can hit it at 14s:
>
> $ time curl http://localhost:/?timer=14
> {"message":"waited 14 seconds"}
> real 0m14.019s
> user 0m0.005s
> sys 0m0.005s
>
>
> And at 16s:
>
> $ time curl http://localhost:/?timer=16
> {"message":"waited 16 seconds"}
> real 0m16.018s
> user 0m0.005s
> sys 0m0.005s
>
>
> I wrote a Mojolicious proxy for the slow service:
>
> use Mojolicious::Lite;
>
> get '/slow' => sub {
> my $c = shift;
> my $timer = $c->param('timer') // 10;
> $c->render_later;
>
> my $url = Mojo::URL->new('http://localhost:');
> $url->query(timer => $timer);
>
> app->log->debug("Invoking " . $url->to_string);
>
> $c->ua->inactivity_timeout(60)->connect_timeout(60)->request_timeout(60)->get($url,
>
> sub {
> my $tx = pop;
> $c->render(json => $tx->res->json);
> });
> };
>
> app->start;
>
>
> You can see my various *_timeout() invocations on the user agent. If I hit
> the proxy, I get what I expect at 14s:
>
> $ curl --include http://localhost:3000/slow?timer=14
> HTTP/1.1 200 OK
> Content-Length: 31
> Server: Mojolicious (Perl)
> Date: Tue, 05 Sep 2017 20:00:04 GMT
> Content-Type: application/json;charset=UTF-8
>
> {"message":"waited 14 seconds"}
>
>
> But at 16s I get a timeout error:
>
> $ curl --include http://localhost:3000/slow?timer=16
> curl: (52) Empty reply from server
>
>
> Indeed, the proxy's log shows the request and then a timeout 15s after the
> request:
>
> [Tue Sep 5 14:00:45 2017] [debug] Invoking http://localhost:?timer=16
> [Tue Sep 5 14:01:00 2017] [debug] Inactivity timeout
> Mojo::Reactor::Poll: I/O watcher failed: Connection already closed at
> utils/slow-wrap line 15.
>
>
> There's something magic about 15s. I added the various timeouts on the
> user agent, so something else is timing out. I suspect it may be the
> default timeout in the Mojo::IOLoop::Stream constructor:
>
> sub new { shift->SUPER::new(handle => shift, buffer => '', timeout => 15) }
>
>
> If I start my proxy with MOJO_INACTIVITY_TIMOUT=60, then the proxy does
> *not* timeout:
>
> $ curl --include http://localhost:3000/slow?timer=16
> HTTP/1.1 200 OK
> Content-Length: 31
> Server: Mojolicious (Perl)
> Content-Type: application/json;charset=UTF-8
> Date: Tue, 05 Sep 2017 20:04:50 GMT
>
> {"message":"waited 16 seconds"}
>
>
> And the proxy logs are clean:
>
> $ MOJO_INACTIVITY_TIMEOUT=60 ct utils/slow-wrap daemon
> [Tue Sep 5 14:04:28 2017] [info] Listening at "http://*:3000;
> Server available at http://127.0.0.1:3000
> [Tue Sep 5 14:04:34 2017] [debug] GET "/slow"
> [Tue Sep 5 14:04:34 2017] [debug] Routing to a callback
> [Tue Sep 5 14:04:34 2017] [debug] Invoking http://localhost:?timer=16
> [Tue Sep 5 14:04:50 2017] [debug] 200 OK (16.012817s, 0.062/s)
>
>
> My question is: how can I determine what timeout is being triggered at
> 15s, and how can I change it on a per-request basis? I don't want to set
> that timeout for the entire proxy (I do want other slow calls to
> timeout)—just this particular controller. Is that possible? I've seen
> various email threads that recommend adding something like to the
> controller (before invoking the user-agent):
>
> Mojo::IOLoop->stream($c->tx->connection)->timeout(60);
>
>
> which seems to work in this situation, but I'm unsure this is the right
> approach. Is there a better mechanism for this? Will this hold up for many
> requests in the event loop at once?
>
> Scott
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Question about MOJO_INACTIVITY_TIMEOUT effects and Mojo::IOLoop::Stream
I should add that this is Mojolicious 7.43.
On Tuesday, September 5, 2017 at 2:21:31 PM UTC-6, Scott Wiersdorf wrote:
>
> I have a slow HTTP service I'm writing a proxy for. If I hit the service
> directly using curl, it looks like this:
>
> $ time curl http://localhost:/?timer=10
> {"message":"waited 10 seconds"}
> real 0m10.018s
> user 0m0.005s
> sys 0m0.005s
>
>
> I can hit it at 14s:
>
> $ time curl http://localhost:/?timer=14
> {"message":"waited 14 seconds"}
> real 0m14.019s
> user 0m0.005s
> sys 0m0.005s
>
>
> And at 16s:
>
> $ time curl http://localhost:/?timer=16
> {"message":"waited 16 seconds"}
> real 0m16.018s
> user 0m0.005s
> sys 0m0.005s
>
>
> I wrote a Mojolicious proxy for the slow service:
>
> use Mojolicious::Lite;
>
> get '/slow' => sub {
> my $c = shift;
> my $timer = $c->param('timer') // 10;
> $c->render_later;
>
> my $url = Mojo::URL->new('http://localhost:');
> $url->query(timer => $timer);
>
> app->log->debug("Invoking " . $url->to_string);
>
> $c->ua->inactivity_timeout(60)->connect_timeout(60)->request_timeout(60)->get($url,
>
> sub {
> my $tx = pop;
> $c->render(json => $tx->res->json);
> });
> };
>
> app->start;
>
>
> You can see my various *_timeout() invocations on the user agent. If I hit
> the proxy, I get what I expect at 14s:
>
> $ curl --include http://localhost:3000/slow?timer=14
> HTTP/1.1 200 OK
> Content-Length: 31
> Server: Mojolicious (Perl)
> Date: Tue, 05 Sep 2017 20:00:04 GMT
> Content-Type: application/json;charset=UTF-8
>
> {"message":"waited 14 seconds"}
>
>
> But at 16s I get a timeout error:
>
> $ curl --include http://localhost:3000/slow?timer=16
> curl: (52) Empty reply from server
>
>
> Indeed, the proxy's log shows the request and then a timeout 15s after the
> request:
>
> [Tue Sep 5 14:00:45 2017] [debug] Invoking http://localhost:?timer=16
> [Tue Sep 5 14:01:00 2017] [debug] Inactivity timeout
> Mojo::Reactor::Poll: I/O watcher failed: Connection already closed at
> utils/slow-wrap line 15.
>
>
> There's something magic about 15s. I added the various timeouts on the
> user agent, so something else is timing out. I suspect it may be the
> default timeout in the Mojo::IOLoop::Stream constructor:
>
> sub new { shift->SUPER::new(handle => shift, buffer => '', timeout => 15) }
>
>
> If I start my proxy with MOJO_INACTIVITY_TIMOUT=60, then the proxy does
> *not* timeout:
>
> $ curl --include http://localhost:3000/slow?timer=16
> HTTP/1.1 200 OK
> Content-Length: 31
> Server: Mojolicious (Perl)
> Content-Type: application/json;charset=UTF-8
> Date: Tue, 05 Sep 2017 20:04:50 GMT
>
> {"message":"waited 16 seconds"}
>
>
> And the proxy logs are clean:
>
> $ MOJO_INACTIVITY_TIMEOUT=60 ct utils/slow-wrap daemon
> [Tue Sep 5 14:04:28 2017] [info] Listening at "http://*:3000;
> Server available at http://127.0.0.1:3000
> [Tue Sep 5 14:04:34 2017] [debug] GET "/slow"
> [Tue Sep 5 14:04:34 2017] [debug] Routing to a callback
> [Tue Sep 5 14:04:34 2017] [debug] Invoking http://localhost:?timer=16
> [Tue Sep 5 14:04:50 2017] [debug] 200 OK (16.012817s, 0.062/s)
>
>
> My question is: how can I determine what timeout is being triggered at
> 15s, and how can I change it on a per-request basis? I don't want to set
> that timeout for the entire proxy (I do want other slow calls to
> timeout)—just this particular controller. Is that possible? I've seen
> various email threads that recommend adding something like to the
> controller (before invoking the user-agent):
>
> Mojo::IOLoop->stream($c->tx->connection)->timeout(60);
>
>
> which seems to work in this situation, but I'm unsure this is the right
> approach. Is there a better mechanism for this? Will this hold up for many
> requests in the event loop at once?
>
> Scott
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Question about MOJO_INACTIVITY_TIMEOUT effects and Mojo::IOLoop::Stream
I have a slow HTTP service I'm writing a proxy for. If I hit the service
directly using curl, it looks like this:
$ time curl http://localhost:/?timer=10
{"message":"waited 10 seconds"}
real 0m10.018s
user 0m0.005s
sys 0m0.005s
I can hit it at 14s:
$ time curl http://localhost:/?timer=14
{"message":"waited 14 seconds"}
real 0m14.019s
user 0m0.005s
sys 0m0.005s
And at 16s:
$ time curl http://localhost:/?timer=16
{"message":"waited 16 seconds"}
real 0m16.018s
user 0m0.005s
sys 0m0.005s
I wrote a Mojolicious proxy for the slow service:
use Mojolicious::Lite;
get '/slow' => sub {
my $c = shift;
my $timer = $c->param('timer') // 10;
$c->render_later;
my $url = Mojo::URL->new('http://localhost:');
$url->query(timer => $timer);
app->log->debug("Invoking " . $url->to_string);
$c->ua->inactivity_timeout(60)->connect_timeout(60)->request_timeout(60)->get($url,
sub {
my $tx = pop;
$c->render(json => $tx->res->json);
});
};
app->start;
You can see my various *_timeout() invocations on the user agent. If I hit
the proxy, I get what I expect at 14s:
$ curl --include http://localhost:3000/slow?timer=14
HTTP/1.1 200 OK
Content-Length: 31
Server: Mojolicious (Perl)
Date: Tue, 05 Sep 2017 20:00:04 GMT
Content-Type: application/json;charset=UTF-8
{"message":"waited 14 seconds"}
But at 16s I get a timeout error:
$ curl --include http://localhost:3000/slow?timer=16
curl: (52) Empty reply from server
Indeed, the proxy's log shows the request and then a timeout 15s after the
request:
[Tue Sep 5 14:00:45 2017] [debug] Invoking http://localhost:?timer=16
[Tue Sep 5 14:01:00 2017] [debug] Inactivity timeout
Mojo::Reactor::Poll: I/O watcher failed: Connection already closed at
utils/slow-wrap line 15.
There's something magic about 15s. I added the various timeouts on the
user agent, so something else is timing out. I suspect it may be the
default timeout in the Mojo::IOLoop::Stream constructor:
sub new { shift->SUPER::new(handle => shift, buffer => '', timeout => 15) }
If I start my proxy with MOJO_INACTIVITY_TIMOUT=60, then the proxy does
*not* timeout:
$ curl --include http://localhost:3000/slow?timer=16
HTTP/1.1 200 OK
Content-Length: 31
Server: Mojolicious (Perl)
Content-Type: application/json;charset=UTF-8
Date: Tue, 05 Sep 2017 20:04:50 GMT
{"message":"waited 16 seconds"}
And the proxy logs are clean:
$ MOJO_INACTIVITY_TIMEOUT=60 ct utils/slow-wrap daemon
[Tue Sep 5 14:04:28 2017] [info] Listening at "http://*:3000;
Server available at http://127.0.0.1:3000
[Tue Sep 5 14:04:34 2017] [debug] GET "/slow"
[Tue Sep 5 14:04:34 2017] [debug] Routing to a callback
[Tue Sep 5 14:04:34 2017] [debug] Invoking http://localhost:?timer=16
[Tue Sep 5 14:04:50 2017] [debug] 200 OK (16.012817s, 0.062/s)
My question is: how can I determine what timeout is being triggered at 15s,
and how can I change it on a per-request basis? I don't want to set that
timeout for the entire proxy (I do want other slow calls to timeout)—just
this particular controller. Is that possible? I've seen various email
threads that recommend adding something like to the controller (before
invoking the user-agent):
Mojo::IOLoop->stream($c->tx->connection)->timeout(60);
which seems to work in this situation, but I'm unsure this is the right
approach. Is there a better mechanism for this? Will this hold up for many
requests in the event loop at once?
Scott
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Mojo under docker
Running hypnotoad under Docker would probably be considered a Docker anti-pattern, since a Linux container is just a single Linux process. In a Docker world you would run a Mojo::Server::Daemon (`myapp daemon`) and let some Docker-level scheduler manage the individual instances (e.g., Kubernetes, Mesos, etc.), managing the scaling and being responsible for starting more instances when they die—which is roughly the same area of concern as hypnotoad. If you don't have access to a container orchestration system and must run hypnotoad in a container, look at Mojo::Server::Hypnotoad and set HYPNOTOAD_FOREGROUND as Dan recommended. Scott On Wednesday, August 2, 2017 at 2:55:29 PM UTC-6, Daniel Suen wrote: > > I would like to run hypnotoad programmatically under docker, I know the > command line has a -f switch but I don't see it for the > Mojo::Server::Hypnotoad. Does anyone know how to do this? Any help is > appreciated. Thanks. > > Daniel > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Re: Please give me less trivial example of non-blocking Mojo::UserAgent
I agree with Alexander Karelas: your question is a textbook job queue
application.
It is possible to solve this using only non-blocking user agents, but in
either case you'll have to write some kind of controller that keeps track
of "in-flight" requests, a queue of pending requests, and something to
manage the queue. It's not a trivial application to do it well—you're
essentially implementing Minion. This is likely not the right forum to ask
for someone to write this for you.
Scott
On Monday, November 14, 2016 at 5:53:50 AM UTC-7, Alex Povolotsky wrote:
>
> I do not need an extra job manager with forks, I need to employ Mojo's
> event-based nonblocking I/O
>
> On Mon, Nov 14, 2016 at 3:46 PM, Alexander Karelas <alex.k...@gmail.com
> > wrote:
>
>> I'm not an expert, but what you're asking for looks like a job for
>> Minion: https://metacpan.org/pod/Minion
>>
>> On 14/11/16 14:41, Alex Povolotsky wrote:
>>
>> Looks like I was unclear. I do not need only extract all links, I must
>> continue fetching and parsing them, not more than N at a time and if
>> possible not less. I need some king of mirroring crawler.
>>
>> On Sat, Nov 12, 2016 at 6:25 PM, Scott Wiersdorf <scott.w...@gmail.com
>> > wrote:
>>
>>> Here is a complete working example you can run:
>>>
>>> #!/usr/bin/env perl
>>> use Mojolicious::Lite;
>>>
>>> get '/random-urls' => sub {
>>> my $c = shift;
>>> $c->render_later;
>>>
>>> $c->delay(
>>> sub { ## first step
>>> my $delay = shift;
>>>
>>> $c->ua->get('https://www.random.org/bytes/',
>>> $delay->begin);
>>> $c->ua->get('https://www.random.org/integer-sets/',
>>> $delay->begin);
>>> $c->ua->get('https://www.random.org/strings/',
>>> $delay->begin);
>>> $c->ua->get('https://www.random.org/audio-noise/',
>>> $delay->begin);
>>> },
>>>
>>> sub { ## second step
>>> my $delay = shift;
>>>
>>> for my $dom (map { $_->res->dom } @_) {
>>> say STDERR $dom->at('title')->text;
>>> }
>>>
>>> $c->render(text => "Got all the links");
>>> }
>>> );
>>> };
>>>
>>> app->start;
>>>
>>>
>>> When the results finally all come back from the first step, the second
>>> step will print out the page titles to STDERR, and return to the client
>>> "Got all the links".
>>>
>>> Scott
>>>
>>> On Saturday, November 12, 2016 at 2:47:57 AM UTC-7, Alex Povolotsky
>>> wrote:
>>>>
>>>> Hello
>>>>
>>>> Examples of Mojo::UserAgent are limited to fetching a set of files.
>>>>
>>>> But I need a bit more complex thing: I need to parse a site, reading
>>>> pages, parsing it and reading links, using non-blocking UA with, say, 4
>>>> downloads at a time, no more and if possible no less.
>>>>
>>>> Can someone give me a good example?
>>>>
>>>> Alex
>>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Mojolicious" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/mojolicious/wdp_pgd4e0k/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> mojolicious...@googlegroups.com .
>>> To post to this group, send email to mojol...@googlegroups.com
>>> .
>>> Visit this group at https://groups.google.com/group/mojolicious.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Mojolicious" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to mojolicious...@googlegroups.com .
>> To post to this group, send email to mojol...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/mojolicious.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Mojolicious" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/mojolicious/wdp_pgd4e0k/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> mojolicious...@googlegroups.com .
>> To post to this group, send email to mojol...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/mojolicious.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: websocket route that polls a database
On Monday, August 8, 2016 at 9:20:09 PM UTC-6, Aaron wrote:
>
> Is there a way to do the database polling in a single loop that then sends
> the notification out to all connected web socket clients? Does this need
> to be set up as an entirely other application that does the polling and
> then have the web socket route listen for messages from it? Or am I
> overcomplicating this horribly?
>
What about something like (untested):
my %clients = ();
Mojo::IOLoop->recurring(5 => sub {
## poll the db
...
## broadcast to all connected clients
$_->send({text => "updated"}) for values %clients;
});
## routes here
...
When a client connects, store the socket (I believe $ws will be stringified
as the key):
$c->on(message => sub { my $ws = shift; $clients{$ws} = $ws })
and when a client disconnects, remove the socket:
$c->on(finish => sub { my $ws = shift; delete $clients{$ws} });
Scott
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Feature request - Mojo::DOM, have a line number for each element?
You can use map() to do that:
$dom->find('div')->map(sub { state $i = 0; say $i++ . " $_" });
Scott
On Friday, July 22, 2016 at 1:44:45 AM UTC-6, Ekki Plicht wrote:
>
> I use Mojo::DOM for various web scraping and analysis, very easy, very
> fast, nice.
>
> Usually I am interested in only a few tags, not the entire dom. So I use
> ->find() to select the interesting nodes, check some facts on the found
> nodes and store the results in a database for later viewing.
>
> For this later viewing I would love to retain the sequence in which the
> nodes are in the source. Unfortunately all information about the sequence
> of tags is lost when I use ->find().
>
> The parser I used to use before (HMTL::HTML5::Parser) does provide a
> line-number function for each element. This is enough for me to retain the
> sequence of nodes, the absolute position is not important.
>
> Do you think it would be possible to extend Mojo::DOM to provide a line
> number for each element? I understand this this might be insufficient for
> the situation where many tags are on the same line, but that's too bad
> then...
>
> TIA,
> Ekki
>
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] New plugin for Google Cloud Platform APIs
I've pushed an initial working plugin to github that communicates with
Google Cloud Platform APIs:
$c->app->gcp_ua(GET =>
"https://pubsub.googleapis.com/v1/projects/$ENV{GCP_PROJECT}/topics;,
sub { ## response handler
my ($ua, $tx) = @_;
$c->render(json => $tx->res->json, status =>
$tx->res->code);
},
sub { ## error sub
my ($tx, $c) = @_;
$c->render(json => $tx->res->json, status =>
$tx->res->code);
}
);
It makes use of Mojo::JWT::Google to do the hard part. It is not thoroughly
tested, but works for the happy case. I plan on adding it to CPAN tonight,
but would love any feedback first.
https://github.com/scottw/p5-Mojolicious-Plugin-Google-Cloud-UserAgent
Scott
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Posting with useragent after rendering
Is there a reason you can't do something like this?:
sub res1{
my $c=shift;
$c->ua->post('http://www.example.com', sub {
my ($uax, $txx) = @_;
$c->app->log->debug(Dumper($txx));
});
$c->render(status=>200,data=>'');
}
The way you're doing it, the transaction goes out of scope which is
probably why you get the premature connection close. In the above example,
the post() will return immediately, allowing the render() to execute next.
The post() response will be handled asynchronously whenever it comes back.
Scott
On Tuesday, April 26, 2016 at 5:57:17 PM UTC-6, Iván Aponte wrote:
>
> Hello,
>
> I have the following problem. I have a service which gets call, it must
> answer with 200, close the connection and post to another web service. If I
> do it in a non-blocking way premature closing occurs.
>
> E.g. :
>
> sub res1{
> my $c=shift;
>$c->render(status=>200,data=>'');
>my $ua = Mojo::UserAgent->new;
>my $tx = $ua->build_tx(POST => 'http//www.example.com');
>$ua->start($tx=> sub{
> my ($uax, $txx) = @_;
> $c->app->log->debug(Dumper($txx));
>});
> }
>
>
>
> Is there a way to make a post after rendering without blocking ?
>
>
> Thanks,
>
> IA
>
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
[Mojolicious] Re: Cutesy dates (going full stack or not)
On Wednesday, August 27, 2014 9:03:21 PM UTC-6, sri wrote: The feature proposal in this branch may seem rather simple, but it represents a possible new chapter for Mojolicious. https://github.com/kraih/mojo/compare/time_in_words We are not at the point where we are going to select a blessed model layer. But what we could do is shift away a bit from trying to provide only the absolute essentials, and start embracing more (and higher level) established best practices from outside the current Perl eco system, such as cutesy dates. What do you think, where should we draw the line when it comes to high level features? I don't know if there is such a line as long as there is a compelling need. A rule of thumb I (try to) follow with my own software is would this delight people? If Mojolicious had a tagline, it would be Look how simple it is to do amazing things! I think any feature that supports that is worth consideration. With the cutesy dates specifically, however, we would probably want to make it I18N-able, which could open up a worm can, since (I think...) this would represent the first Mojo::* whose content isn't configurable (e.g. with a template). Is that right? Besides that small issue, I don't have a personal need for this kind of thing (I usually reach for Time::Piece these days), but others may have a stronger opinion. -- You received this message because you are subscribed to the Google Groups Mojolicious group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at http://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
Re: [Mojolicious] Nested helpers
On Saturday, August 9, 2014 6:25:05 AM UTC-6, sri wrote:
I'm afraid the nested_helpers proposal didn't get enough votes and had to
be rejected. :(
I'm a late-comer to the party I realize, but I'm not in favor of the
proposed solution either. It feels and looks like a hack, and is barely
better than longer helper names. Lately I've been grouping my helpers into
classes with their own namespaces, then creating an instance of the class
and invoking it like an object. Something like this:
package My::Plugin::Frob;
use Mojo::Base 'Mojolicious::Plugin';
use Frob; ## the helper class
has 'frob'; ## this plugin's attribute
sub register {
my ($self, $app, $cfg) = @_;
$self-frob(Frob-new($cfg)); ## create an instance and assign to the
attribute
$app-helper(frob = sub { $self-frob }); ## make the helper
...
}
then elsewhere, say, in a controller:
$c-frob-some_method(...);
If there's a cleaner way to do this with less boilerplate, I'd be in favor
of that too, but this has worked very well for over a year.
Scott
--
You received this message because you are subscribed to the Google Groups
Mojolicious group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at http://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
