There is no mechanism for automatic load balancing across the board; some individual operators support it (e.g. Kafka input operator) out of the box. You can implement it in your own operator by using a custom partitioner.
Keys are sticky by default since the hashcode is used to distribute tuples to partitions. There is no special setting for this. You can gain more control over how the distribution happens by using a StreamCodec. Ram On Fri, Jun 10, 2016 at 11:36 AM, Junguk Cho <[email protected]> wrote: > Hi, Ram. > > Thank you for reply. > I have following question. > > 1. Where do we specify policy to use "Load balance" or "Sticky Key"? > > 2. This is just general question. > For example, I would like to run A class now as one operator and then > later I need other tasks which I do not think. > At that time, could I load this new class in current DAG even though it is > not included in *.apa file. > > > Thanks, > Junguk > > > 2016-06-10 12:03 GMT-04:00 Munagala Ramanath <[email protected]>: > >> You don't need dynamic partitioning to achieve that topology. You can >> simply create your DAG as: A --> X --> Y and then set the *PARTITIONER* >> attribute on X >> as discussed in the "Advanced Features" section of the TopN words >> tutorial at: >> http://docs.datatorrent.com/tutorials/topnwords-c7/ >> >> The stanza in your properties file should look something like this (with >> suitable changes >> for the application and operator names): >> >> *<property>* >> * >> <name>dt.application.TopNWordsWithQueries.operator.wordReader.attr.PARTITIONER</name>* >> * >> <value>com.datatorrent.common.partitioner.StatelessPartitioner:2</value>* >> *</property>* >> >> The platform will then create two replicas of X. >> >> For the rest, could you provide additional details of what you are trying >> to accomplish, what >> you tried and what sort of problems you encountered ? That will help us >> provide more precise answers. >> >> Ram >> >> On Fri, Jun 10, 2016 at 8:37 AM, Junguk Cho <[email protected]> wrote: >> >>> Hi. >>> >>> I explore "Dynamic Application Modification" with this example( >>> https://github.com/DataTorrent/examples/tree/master/tutorials/dynamic-partition >>> ). >>> >>> >>> To be clear, basically, application developers carefully design a >>> program. >>> For Dynamic Partitioning, in "dynamic partition" example, it uses >>> threadhold to adjust partition. In addition, to use Modification of DAG >>> structure, app developers should make program like java classes to use them >>> later. >>> Do I interpret that they are determined "compiling time"? >>> >>> How do I use "Load balance and Sticky Key" in Apex? >>> I would like to create this topology for word counting. >>> For example, >>> B operator >>> A operator -> -> one >>> onperator >>> C Operator >>> >>> A sends the same values like "word" to the same operator ( B or C )? >>> Can we specify this rule in code and how to make two process for B and C >>> with one class? >>> >>> Thanks, >>> Junguk >>> >> >> >
