Hi, this is my first post here. But i play with flume for some time now. My question is how well flume scale? Can Flume ingest +100k events per seccond? Has anyone tried something like this?
I created simple test and results are really slow. I wrote simple app with rpc client with fallback using flume sdk which is reading dummy log file. In the end i have two flume agents which are writing to hdfs. rollInterval = 60 And in hdfs i get files with ~12MB. Do i need to use some complex topology with 3 tier? How many flume agents should write to hdfs? Best regards.
