On 08/20/2015 06:53 PM, Timur Batyrshin wrote:
Hi,
Is there a way to set messagematcher for an output dynamically and have
that set on per instance of connection basis?
This would be possible, yes, but doesn't yet exist. If you were going to be
doing it in Heka, you'd need to write a new output plugin that waited for some
consumer to make a connection and dynamically spun up a new output instance of
some other type with the specified message matcher. It would probably end up
looking a lot like the SandboxManagerFilter, which can dynamically add and
remove SandboxFilter plugins to the router based on control messages. I think
this is an interesting feature and would definitely consider merging such an
output to the core.
I’m looking into a use case when I have a “central” Heka server to which
connect remote clients and need to have routed
only part of the traffic to them. I could simply use messagematcher on
the client side but this will require them to receive all
the messages which can be a large amount. So I thought if a client was
able to set messagematcher for his connection while
connecting to central heka.
The usecase of that would be a various debug/research activities done on
the production stream of events without affecting it.
For example, human polling for specific types of events which he expects
to see for debugging prod issues or it could be
playing with heka on real stream of events to develop working
configuration for further deployment.
Any ideas on if I can do that?
Not just yet. The best we have so far is the SandboxManagerFilter. If you
correctly set up a TcpInput for receiving signed control messages and a
SandboxManagerFilter which expects those messages then you can let people
inject a new filter (with a custom message matcher) into the running Heka. You
could either set that up on your central Heka server, or you could forward all
of the central server's traffic to a different Heka used for this purpose.
Your users wouldn't then each have their own Heka, making connections to the
central Heka server. Instead, they'd write filters to do whatever they need and
then shoot them into the shared Heka, able to send output to the shared Heka's
dashboard.
Hope this helps,
-r
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
