Hi there Paul,
Firstly, your use case looks really interesting and hope to see a few
more posts on how you use Geode further. Keep us informed we like to
hear what you guys are doing with GEODE! :)
The subscription or CQ (continuous query) paradigm is, as stated, a 1:1
relationship. When a client registers interest on a region that client
will be notified. This is more of a topic semantic rather than a queue
semantic.
Although this is not the first time I've heard the request for this kind
of functionality. To best explain why GEODE, currently, implements the
1:1 relationship has got to do with guaranteed delivery and in-order
delivery. If we use a queue semantic, with multiple clients being able
to process data in a balanced manner, we end up with potential
out-of-order processing of messages. In addition to that it now becomes
significantly harder to track and n deal with client failures and the
potential replaying of messages.
But that said, I have seen other users resolve this problem and could
detail some approaches in a later correspondence if you'd like.
--Udo
On 1/19/17 03:39, Paul Perez wrote:
Hello All,
As explained in a previous email, we try to use Geode to process and
aggregate a stream of Traces. Our requirement is to process billions
of simple traces every day.
We imagine the aggregation process in many steps.
One: traces are generated by a tiers tools and stored in a first geode
region
Two: once a trace put in the first region we use the async event
feature to invoke a client that executes the first aggregation steps.
Then the result will be put in a second region.
Three: the second aggregation step is in the same way, when traces are
put in the second region, then an asynchronous event is sent to the
client to execute the second part of the aggregation etc.…
For scalability purposes, we plan to use many clients that could
receive the events and execute the aggregation and put the results
back to Geode.
Consequently, as far as we understand the documentation, when an entry
is put in a region, each client that registered an interest receives
an event and aggregate the trace. So, the trace will be aggregated
many times.
My question is: If many clients are registered, could we configure
the region to send randomly, the event to one client only.
A subsidiary question: Do we have the same behaviour with the function
execution feature or it could be an alternative in that case
Thank you for your help
Best regards
Paul
