[Mojolicious] Re: Need help (feedback) with Mojo::Redis2 - A new pure-perl non-blocking I/O Redis driver
This makes a new connection on each request:
helper redis = sub { shift-stash-{redis} ||= Mojo::Redis2-new(url =
$redis_server)};
On Saturday, October 4, 2014 2:46:56 AM UTC+2, Daniel Mantovani wrote:
I was trying to handle several subscribe channels with only one redis
client using Mojo::Redis2, but it seems that I'm doing something wrong, I
end up using as many clients as open subscribed channels I have.
Maybe somebody can take a look at this test code and let me know :
https://github.com/dmanto/test-redis-connections.git
Thanks in advance,
El martes, 5 de agosto de 2014 11:04:45 UTC-3, Jan Henning Thorsen
escribió:
I pushed an experimental release to CPAN now:
https://metacpan.org/pod/Mojo::Redis2
-Any- feedback is more than welcome.
On Sunday, June 29, 2014 10:21:28 PM UTC+2, Daniel Mantovani wrote:
Perfect then, thanks Jan
El domingo, 29 de junio de 2014 03:50:09 UTC-3, Jan Henning Thorsen
escribió:
The on(message = $channel = ...) API is not going to be part of
Mojo::Redis2. It will just be on(message = sub { my ($self, $message,
$channel) = @_ }); The reason for that is to have a consistent API. (on()
is already defined in Mojo::EventEmitter)
And yes, it will only have one connection to the Redis server pr.
$redis object, even though you do $redis-subscribe(channel:$_) for
1..1;
I'm planning to have one connection for the pubsub
http://redis.io/commands#pubsub commands, a new connection when you
call blpop, brpop, brpoplpush http://redis.io/commands#list, and one
connection for everything else. The reason for creating a new connection
on
each blpop(), brpop(), brpoplpush http://redis.io/commands#list() is
that they are blocking on the server side.
On Saturday, June 28, 2014 2:12:01 AM UTC+2, Daniel Mantovani wrote:
Yes it absolutly matters, I need a different channel per connected
websocket
So if I understood correctly inside my websocket route I could write
something like:
my $applianceid = ... (some unique identifier for each appliance, I
get it from a custom header)
my $redis = Mojo::Redis2-new;
$redis-on(message = myapp:$applianceid, sub {
my ($rself, $msg) = @_;
.
do something with the received redis msg
.
$tx-send($something); # use my transaction object to send some data
to my connected appliance
etc
});
...
and still use just one redis client despite I may have thousands of
websocket clients connected and subscribed to different channels
(myapp:$applianceid)?
In this case I think the solution would be perfect, I don´t care
using one extra redis client for publishing, that I guess I will be able
to
share among all websockets.
So at the end of the day I will be using just two redis clients per
toad when running on hypnotoad... is that right or I missunderstood?
El viernes, 27 de junio de 2014 10:11:52 UTC-3, Jan Henning Thorsen
escribió:
I had some typos:
I meant subscribing to foo and then bar. (not sure if that
matters for the example though)
Also the $old comments should be now there is two connections to the
Redis database.
On Friday, June 27, 2014 3:09:35 PM UTC+2, Jan Henning Thorsen wrote:
Mojo::Redis2 can (or at least it will be able to) (p)subscribe
multiple times, using just one connection. I will try to illustrate the
difference to be sure I understand you:
my $old = Mojo::Redis-new;
my $s1 = $old-subscribe(foo);
my $s2 = $old-subscribe(foo);
# now you two connections to the Redis database
my $new = Mojo::Redis2-new;
$new-subscribe(foo);
$new-subscribe(foo);
# still just one connection to the Redis database, since
Mojo::Redis2 is re-using the same subscribe connection
Unfortunatly, issuing (P)SUBSCRIBE will only allow subscribe like
commands http://redis.io/commands/subscribe, which renders any
Redis module unable to do PUBLISH and SUBSCRIBE over the same database
connection.
On Friday, June 27, 2014 3:00:20 PM UTC+2, Daniel Mantovani wrote:
Hi Jan, I´ve using Mojo::Redis and find it very convenient as the
glue that allows to comunicate non-blocking among different processes
(toads) when running hypnotoad (using subscribe and publish, just as
the
Websocket example in the module's synopsis).
Something I would suggest for you to take a look is related to the
scalling of simultaneos redis clients needed for a realtime app. I
understand that you would need a dedicated subscribe client plus a
publish per connected websocket, if you follow the mencioned example.
That could be a problem depending on the nature of your app. In my
case I was working on an app that will have thousand of appliances
with
very low traffic but allways on, so I ended up using Mojo ::Redis with
something like a psubscribe app:* client per toad, and just one
publishing client (probably blocking) also per toad (Anyway I'm not
sure
how this solution will scale in the field, we will see
[Mojolicious] Re: Need help (feedback) with Mojo::Redis2 - A new pure-perl non-blocking I/O Redis driver
I was trying to handle several subscribe channels with only one redis
client using Mojo::Redis2, but it seems that I'm doing something wrong, I
end up using as many clients as open subscribed channels I have.
Maybe somebody can take a look at this test code and let me know :
https://github.com/dmanto/test-redis-connections.git
Thanks in advance,
El martes, 5 de agosto de 2014 11:04:45 UTC-3, Jan Henning Thorsen escribió:
I pushed an experimental release to CPAN now:
https://metacpan.org/pod/Mojo::Redis2
-Any- feedback is more than welcome.
On Sunday, June 29, 2014 10:21:28 PM UTC+2, Daniel Mantovani wrote:
Perfect then, thanks Jan
El domingo, 29 de junio de 2014 03:50:09 UTC-3, Jan Henning Thorsen
escribió:
The on(message = $channel = ...) API is not going to be part of
Mojo::Redis2. It will just be on(message = sub { my ($self, $message,
$channel) = @_ }); The reason for that is to have a consistent API. (on()
is already defined in Mojo::EventEmitter)
And yes, it will only have one connection to the Redis server pr. $redis
object, even though you do $redis-subscribe(channel:$_) for 1..1;
I'm planning to have one connection for the pubsub
http://redis.io/commands#pubsub commands, a new connection when you
call blpop, brpop, brpoplpush http://redis.io/commands#list, and one
connection for everything else. The reason for creating a new connection on
each blpop(), brpop(), brpoplpush http://redis.io/commands#list() is
that they are blocking on the server side.
On Saturday, June 28, 2014 2:12:01 AM UTC+2, Daniel Mantovani wrote:
Yes it absolutly matters, I need a different channel per connected
websocket
So if I understood correctly inside my websocket route I could write
something like:
my $applianceid = ... (some unique identifier for each appliance, I get
it from a custom header)
my $redis = Mojo::Redis2-new;
$redis-on(message = myapp:$applianceid, sub {
my ($rself, $msg) = @_;
.
do something with the received redis msg
.
$tx-send($something); # use my transaction object to send some data
to my connected appliance
etc
});
...
and still use just one redis client despite I may have thousands of
websocket clients connected and subscribed to different channels
(myapp:$applianceid)?
In this case I think the solution would be perfect, I don´t care using
one extra redis client for publishing, that I guess I will be able to
share
among all websockets.
So at the end of the day I will be using just two redis clients per
toad when running on hypnotoad... is that right or I missunderstood?
El viernes, 27 de junio de 2014 10:11:52 UTC-3, Jan Henning Thorsen
escribió:
I had some typos:
I meant subscribing to foo and then bar. (not sure if that matters
for the example though)
Also the $old comments should be now there is two connections to the
Redis database.
On Friday, June 27, 2014 3:09:35 PM UTC+2, Jan Henning Thorsen wrote:
Mojo::Redis2 can (or at least it will be able to) (p)subscribe
multiple times, using just one connection. I will try to illustrate the
difference to be sure I understand you:
my $old = Mojo::Redis-new;
my $s1 = $old-subscribe(foo);
my $s2 = $old-subscribe(foo);
# now you two connections to the Redis database
my $new = Mojo::Redis2-new;
$new-subscribe(foo);
$new-subscribe(foo);
# still just one connection to the Redis database, since
Mojo::Redis2 is re-using the same subscribe connection
Unfortunatly, issuing (P)SUBSCRIBE will only allow subscribe like
commands http://redis.io/commands/subscribe, which renders any
Redis module unable to do PUBLISH and SUBSCRIBE over the same database
connection.
On Friday, June 27, 2014 3:00:20 PM UTC+2, Daniel Mantovani wrote:
Hi Jan, I´ve using Mojo::Redis and find it very convenient as the
glue that allows to comunicate non-blocking among different processes
(toads) when running hypnotoad (using subscribe and publish, just as
the
Websocket example in the module's synopsis).
Something I would suggest for you to take a look is related to the
scalling of simultaneos redis clients needed for a realtime app. I
understand that you would need a dedicated subscribe client plus a
publish per connected websocket, if you follow the mencioned example.
That could be a problem depending on the nature of your app. In my
case I was working on an app that will have thousand of appliances with
very low traffic but allways on, so I ended up using Mojo ::Redis with
something like a psubscribe app:* client per toad, and just one
publishing client (probably blocking) also per toad (Anyway I'm not
sure
how this solution will scale in the field, we will see in the
followings
months probably).
Hope you can add to Mojo::Redis2 some kind of support for a
solution that allows to share psubscribe redis connections among many
users
on a single process or toad.
BR,
[Mojolicious] Re: Need help (feedback) with Mojo::Redis2 - A new pure-perl non-blocking I/O Redis driver
Hi Jan, I´ve using Mojo::Redis and find it very convenient as the glue
that allows to comunicate non-blocking among different processes (toads)
when running hypnotoad (using subscribe and publish, just as the Websocket
example in the module's synopsis).
Something I would suggest for you to take a look is related to the
scalling of simultaneos redis clients needed for a realtime app. I
understand that you would need a dedicated subscribe client plus a
publish per connected websocket, if you follow the mencioned example.
That could be a problem depending on the nature of your app. In my case I
was working on an app that will have thousand of appliances with very low
traffic but allways on, so I ended up using Mojo ::Redis with something
like a psubscribe app:* client per toad, and just one publishing client
(probably blocking) also per toad (Anyway I'm not sure how this solution
will scale in the field, we will see in the followings months probably).
Hope you can add to Mojo::Redis2 some kind of support for a solution that
allows to share psubscribe redis connections among many users on a single
process or toad.
BR,
Daniel
El jueves, 26 de junio de 2014 09:24:00 UTC-3, Jan Henning Thorsen escribió:
Hi,
I'm trying to make a new Redis driver for Mojolicious that makes sense. Me
and Marcus Ramberg has been maintaining Mojo::Redis, but I see some issues
with the API:
* No blocking support
* Really poor error handling
* Trying to be clever with BLPOP and SUBSCRIBE, using an overloaded on()
method
* Some really weird bugs that that I can't seem to figure out (Got one
project at work which fail on random)
* Public methods that is confusing: connect(), connected(), disconnect()
(?), protocol_redis() and timeout()
So now I want to make a new version that Makes Sense (tm). The project is
currently just a branch under the old mojo-redis repo:
https://github.com/marcusramberg/mojo-redis/tree/v2
# What I need help with
1. Look if the API makes sense.
2. Look at the code to see if it makes sense.
3. Tell me when something is awkward or plain wrong.
4. Figure out if -execute() makes sense and/or if it could be replace
with something nicer
I often do many commands at once, so what execute() fix is this:
Mojo::IOLoop-delay(
sub {
my ($delay) = @_;
$redis-get(foo = $delay-begin);
$redis-get(bar = $delay-begin);
$redis-get(baz = $delay-begin);
},
sub {
my ($delay, $foo_err, $foo, $bar_err, $bar, $baz_err, $baz) = @_;
# ^ ONE MILLION ARGUMENTS!!
},
);
5. Does it make sense for one $redis object to have multiple connections
to the database?
This is one of the things I built in to the new module: blpop(),
subscribe(), ... will make a new connection to the database instead of user
of the module need to remember which method is blocking or take over the
connection. But does it really make sense?
6. I need a method for UNSUBSCRIBE
The module inherit from Mojo::EventEmitter, so I can't really use the
unsubscribe() method to UNSUBSCRIBE from channels. Got an idea for an
alternative way, or some fancy hack to make it work with unsubscribe()?
I've been considering $redis-unsubscribe(message = $channel_name); but it
kind of breaks the signature.
7. Do Mojo::Redis2 really need encoding attribute?
Why isn't this always UTF-8? I don't get the encoding attribute, but I
kept it because of legacy from the other redis modules.
8. What do you think about pipelined()?
Please have a look at
https://github.com/marcusramberg/mojo-redis/blob/v2/t/pipelined.t
9. ???
# What is done (might be redesigned though)
* I got basic redis methods working blocking and non-blocking
* I got (p)subscribe methods working and (p)message events
* I got pipelining working.
--
You received this message because you are subscribed to the Google Groups
Mojolicious group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at http://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
