The machine that I have seen it both on my machine with 16 GB and 60 GB of memory, when running about 40 clients and ~4k clients respectively using up 100% of memory. If I run without the flume client I have no memory problems, but when I insatiate a flume RPCClient, then I run into memory problems.
Thanks, Matt On Wed, Mar 25, 2015 at 6:42 PM, Hari Shreedharan <[email protected] > wrote: > How much memory are you talking about? The RPC client will hold on to the > batch of events you sent, plus some additional threading overhead. Under > the hood, it uses a Netty client which should not really have a big memory > footprint. > > Thanks, > Hari > > > On Wed, Mar 25, 2015 at 3:27 PM, Matt Fair <[email protected]> wrote: > >> I have an application that launches a bunch of processes (40+) on the >> same machine, each one connects to flume using the default flume >> RPCClient. I however have noticed that each RPCClient takes up a decent >> amount of memory, and when you create as many clients like I am, it adds up >> to a lot of memory. One thought I had to alleviate having to create all of >> the clients was to create only a single RPCClient and then have my other >> processes connect to it via a socket, but that seems a little redundant >> since that is what the RPCClient is suppose to do anyways. Have others >> found themselves in this same situation? Is there a way to handle memory >> more efficiently or is there another RPCClient implementation that doesn't >> take up as much memory? >> >> Thanks, >> Matt >> > >
