The data is written locally on the source node. Both agents will progress independently, and sending data at their own pace to the collectors. They shouldn't affect each other. The UTF8FileTailingAdaptor should track the timestamp of the log files and send it over in time sequence order, hence one failed does not affect others.
regards, Eric On Jul 14, 2011, at 8:45 PM, highpointe wrote: > Ahhhh. Good insight. I didn't think about that. > > Yes, I am aggregating the same data for two clusters. > > So now my challenge is: If one cluster is unreachable... How do I that > cluster's sender fail without affecting the other sender or the connector. > > Excellent insight Ari! Thank you! > > Sent from my iPhone > > On Jul 14, 2011, at 11:54 AM, Ariel Rabkin <[email protected]> wrote: > >> Just to be clear -- you want the same data to end up in each cluster? >> >> You'll have to do some hacking. I think it won't be so very monstrous. >> In the Agent, there's a notion of pluggable Connectors and Senders. I >> think you want a new Connector with two senders. >> >> One caveat -- you'll need to think through what happens if just one >> cluster is down, and perhaps other partial failure scenarios too. >> >> --Ari >> >> On Thu, Jul 14, 2011 at 10:44 AM, high pointe <[email protected]> wrote: >>> I have a situation where I need to take the one data stream, send it through >>> two agents and have the end point residing in separate clusters. >>> Rough explanation.... >>> Streaming Data Box ---> Chukwa Box >>> Agent one on Chukwa Box sends feed to cluster one. >>> Agent two on Chukwa box sends feed to cluster two. >>> Does anyone know of a way to do this without hacking this baby to shreds? >>> >>> Thanks! >> >> >> >> -- >> Ari Rabkin [email protected] >> UC Berkeley Computer Science Department
