Hi all, I am trying to understand how storm's DRPC server and Trident work together. A particular Trident topology I am developing seems to be limited in its throughput as batches/tuples are not pushed into it fast enough from the DRPC server. We are using storm 1.1.0.
Unfortunately I have not found a good source of information on how the DRPC server and Trident function together, though I have looked at the various Trident examples and docs at https://storm.apache.org/releases/1.1.0/index.html as well as Stackoverflow and the old storm Google Group. What I have tried so far: - lowering the trident batch emit interval millis from 500 to 200, 100 and 25, does not seem to have an effect - using multiple workers in the Trident topology along with different kinds of partitioning the stream - seems to hurt performance (I suspect network latency between our storm servers to be the reason for that) - using a single or multiple spouts, seems to have no effect Our drpc is configured as follows: drpc.authorizer.acl.filename "drpc-auth-acl.yaml" drpc.authorizer.acl.strict false drpc.childopts "-Xmx2048m" drpc.http.creds.plugin "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin" drpc.http.port 3774 drpc.https.keystore.password "" drpc.https.keystore.type "JKS" drpc.https.port -1 drpc.invocations.port 3773 drpc.invocations.threads 64 drpc.max_buffer_size 10485760 drpc.port 3772 drpc.queue.size 128 drpc.request.timeout.secs 600 drpc.worker.threads 64 I appreciate any pointers to a good documentation or concrete suggestions. Sorry if this question has been asked before. Thanks Jonas
smime.p7s
Description: S/MIME cryptographic signature
