Re: Runtime Execution Model

Hi Bruno, AFAIK, there is no existing JobFactory that brings as many threads as the partition number. But I think nothing stops you to implement this: you can get the partition information from the JobCoordinator, and then bring as many threads as the partition/task number. Since the two local fa

YARN scheduler configuration

We are currently running Samza on a YARN grid. We unintentionally got into a situation where we needed more capacity than was available (according to YARN), and we found that replacement container requests would just sit there indefinitely waiting to be fulfilled rather than failing. Our monito

Re: Runtime Execution Model

Thanks Yan for writing me back, That's ok for ThreadJobFactory and ProcessJobFactory but what about the YarnJobFactory? How many task/executors will be spawning? Bruno On Mon, Sep 14, 2015 at 7:08 PM, Yan Fang wrote: > Hi Bruno, > > AFAIK, there is no existing JobFactory that brings as many t

Re: Runtime Execution Model

Hi, Bruno, The number of containers are configurable in YarnJobFactory via yarn.container.count. Each container is a single threaded model and you can run multiple tasks in a single container. At maximum, you can have as many containers as the number of tasks in this config to achieve 1 task / thr

Re: Runtime Execution Model

Hi Yi, Does a single task consume from a single partition or it consumes from more/all partitions? Thanks Bruno > On 14 Sep 2015, at 23:22, Yi Pan wrote: > > Hi, Bruno, > > The number of containers are configurable in YarnJobFactory via > yarn.container.count. > Each container is a single th

Re: Review Request 38296: SAMZA-341: Support metrics report via Ganglia

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38296/#review98895 --- build.gradle (line 219)

Re: Runtime Execution Model

Hi, Bruno, The number of partitions consumed by a single task is also configurable via the partition assignment policies (job.systemstreampartition. grouper.factory). By default, there are two partition assignment policies implemented: org.apache.samza.container.grouper.stream.GroupByPartitionFact

Re: Runtime Execution Model

Many thanks Yi Bye Bruno > On 14 Sep 2015, at 23:49, Yi Pan wrote: > > Hi, Bruno, > > The number of partitions consumed by a single task is also configurable via > the partition assignment policies (job.systemstreampartition. > grouper.factory). By default, there are two partition assignment p