Hi,
I'm having a problem that I've tried a lot, and I can not find an answer.
I'm having slowness problem at the moment reading Kafka's data with
Trident. For example in about 10 minutes were issued in the spout less than
120 thousand messages.
BrokerHosts hosts = new ZkHosts(Configuration.ZOOKEEPER_CONNECTION_STRING);
TridentKafkaConfig spoutConf = new TridentKafkaConfig(hosts, topicName);
spoutConf.scheme = new SchemeAsMultiScheme(new StringScheme());
spoutConf.ignoreZkOffsets = true;
spoutConf.startOffsetTime = -1;
spoutConf.bufferSizeBytes = 1024 * 1024 * 4;
spoutConf.fetchSizeBytes = 1024 * 1024 * 4;
OpaqueTridentKafkaSpout spout = new OpaqueTridentKafkaSpout(spoutConf);
Stream stream = topologyTrident.newStream("lines",
spout).parallelismHint(8);
--
Thomas Cristanis
