Agreed with Roshan, VIP is best choice in present. If you want solution
based on Zookeeper, you should wrap to resolve notification and dynamic
configuration by yourself. Much better if someone can try to post patch for
this, not only load balance RPC client, aren't you?
-Regards
Denny Ye
e channel. This
> is load balancing, which is what you are seeking in your scenario; the load
> balancer is not for failover (in the setup of primary and backup servers),
> although there is a FailoverSinkProcessor for if that's needed.
>
> - Connor
>
>
> On Wed, Jan 9, 2
events coming from
'localFlumeA', and load can be dispatched averagely for that two central
Flume servers.
Can it be configured by LoadBalancingSinkProcessor in your mind? Wish
your advice
-Regards
Denny Ye
2013/1/10 Hari Shreedharan
> The LoadBalancing capability similar to the Loa
S : I heard that OS has demon thread to flush page cache to
disk asynchronously with second latency, does it's effective for amount of
data with tolerant loss?
-Regards
Denny Ye
2012/10/22 Jagadish Bihani
> Hi
>
> I am writing this on top of another thread where there was disc
laim for
direct memory, thanks Brock for your tips
-Denny Ye
2012/10/11 Brock Noland
> The only issue with using Direct Memory is that it is not reclaimed
> until the object is garbage collected. So it's possible to run out of
> Direct Memory without actually even garbage collecting
the direct memory allocation with pre-allocated direct memory
blocks. This is more complex than method mentioned above.
Did I explain my thought clearly?
-Regards
Denny Ye
2012/10/11 Senthilvel Rangaswamy
> Denny,
>
> How to do use direct memory ?
>
> Thanks,
> Senthil
>
>
ried to use direct memory for storing
event body out of heap in the past, that's effective method and no any GC
happened. I think you can test it by yourself.
-Regards
Denny Ye
2012/10/11 Brock Noland
> Yep, sounds like: agent heap size < (capacity * avg event size)
>
> Brock
&
'com.hadoop.compression.lzo.LzoCodec' is one of extension for
'org.apache.hadoop.io.compress.CompressionCodec'
2012/8/28 Denny Ye
> hi Kevin,
> I applied for LZO successfully. I will post my LZO configuration, you
> can compare the difference.
>
>
com.hadoop.compression.lzo.LzoCodec
-Regards
Denny Ye
2012/8/28 Kevin Lee
> Folks,
>
> I was follow this link Hadoop at Twitter (part 1): Splittable LZO
> Compression<http://www.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/%5D>
>
20MB/min or 20MB/sec?
I doubt that it may have presentation mistake. Can you confirm it?
-Regards
Denny Ye
2012/8/28 Shara Shi
> Hi Denny
>
> ** **
>
> The throughput is 45MB/sec is OK for me .
>
> But I just got 20M / Minutes
>
> What’s
hi Shara,
You are using MemoryChannel as repository. I tested it with outcomes:
45MB/sec without full GC in local updated code. Is this your goal? or more
high throughput?
-Regards
Denny Ye
2012/8/27 Shara Shi
> Hi All,
>
> ** **
>
> Whatever I have tuned parameters o
Ye
2012/8/26 Mohit Anchlia
>
>
> On Sun, Aug 26, 2012 at 6:47 AM, Denny Ye wrote:
>
>> hi Mohit,
>> Why you confirm it doesn't work at time? I think it reaches to size
>> limitation of your setting 'hdfs.rollSize'. Each snappy file almost 5
&
/flume/2012/08/24/13/dslg2/web.1345840668738.snappy
-rwxr-xr-x 3 root root 407757832 2012-08-24 13:44
/flume_vol/flume/2012/08/24/13/dslg2/web.1345840668737.snappy
-rwxr-xr-x 3 root root 51085873 2012-08-24 13:36
/flume_vol/flume/2012/08/24/13/dslg2/web.1345840465501.snappy
-Regards
Denny
hi David, normally, Flume doesn't write event header to HDFS. Also, you
can expand function what you want [?]
2012/8/21 David Capwell
> I was wondering if I pass random data to an event's header, can the
> HDFSSink write it to HDFS? I know it can use the headers to split the data
> into differ
gt; definitely like to get such a patch committed.
>
> Thanks
> Hari
>
> --
> Hari Shreedharan
>
>
> On Thursday, August 2, 2012 at 8:02 PM, Denny Ye wrote:
>
> > hi all,
> > I posted performance of MemoryChannel last week. That's normal
> throughp
hi Mohit,
Concrete object has been created and invoked by SinkRunner in lifecycle.
-Regards
Denny Ye
2012/8/3 Mohit Anchlia
> Trying to understand how flume sink invocation works. I see HBaseSink.java
> has start and stop, is this object created once or is this object created
>
like 'recoverLease'
interface. 2. Does that file name can be reuse in latest path pattern?
No matter which case, we hope that there is unified activity in path
pattern. Just like your mention, I agree with you. Need some other guys to
discuss may be.
-Regards
Denny Ye
2012/7/31 Yongc
limited
consuming speed in Agent1.
I took some comments in your original mail, wish your attention.
I'm going on tuning in FileChannel, and making increasing throughput
already. Those tuning points I will submit to JIRA later.
-Regards
Denny Ye
2012/7/31 Raymond Ng
> good day al
18 matches
Mail list logo
