[GitHub] xinyuiscool commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

xinyuiscool commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r261027089 ## File path: samza-core/src/main/java/org/apache/samza/util/JobConfigUtil.java ## @@ -0,0 +1,67 @@ +/*

[GitHub] xinyuiscool commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

xinyuiscool commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor URL: https://github.com/apache/samza/pull/920#discussion_r261022606 ## File path: samza-core/src/main/java/org/apache/samza/execution/JobPlanner.java ## @@ -82,10 +84,13 @

[GitHub] rmatharu commented on issue #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

rmatharu commented on issue #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#issuecomment-468111012 Addressed all comments. This is an automate

[GitHub] rmatharu commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

rmatharu commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r261025331 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageM

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260972229 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260978075 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260974797 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260980336 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260973315 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260966831 ## File path: samza-core/src/main/java/org/apache/samza/storage/TaskSideInputSt

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260980813 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260979620 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260976980 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

[GitHub] mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager

mynameborat commented on a change in pull request #912: SEP-19 : Refactoring sideInputs from SamzaContainer to ContainerStorageManager URL: https://github.com/apache/samza/pull/912#discussion_r260972792 ## File path: samza-core/src/main/scala/org/apache/samza/storage/ContainerStora

Re: "send to" ordering is inconsistent

Prateek Maheshwari writes: Hi Tom, I'm assuming that the two sub-DAGs you're talking about are the two Map -> Send To chains acting on the "audit-report-requests" input and sending their results to the "audit-report-status" output. Yes, that's correct. Although processing within each T

Re: "send to" ordering is inconsistent

I was just looking to resolve this by using KafkaProducer's transaction API, assuming the underlying issue is just the async, unordered nature of actual producer sends. I turned on idempotent mode via producer config, which (as expected) reduced the failure rate but didn't eliminate it. Actual se

Re: "send to" ordering is inconsistent

Hi Tom, I'm assuming that the two sub-DAGs you're talking about are the two Map -> Send To chains acting on the "audit-report-requests" input and sending their results to the "audit-report-status" output. Although processing within each Task is in-order, the framework does not guarantee the order

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260926132 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260923638 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260920830 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260920529 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260920066 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260917527 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260919089 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260918760 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260917921 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260913111 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260911441 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260897021 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260888905 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260911377 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260895392 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

rmatharu commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260911400 ## File path: samza-core/src/main/java/org/apache/samza/clusterma

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260892591 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260894484 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260885498 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r259467856 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r259513498 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r260884878 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r259467508 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover

vjagadish1989 commented on a change in pull request #903: SEP-19: Allocator changes for standby-aware container allocation, and active container failover URL: https://github.com/apache/samza/pull/903#discussion_r259489855 ## File path: samza-core/src/main/java/org/apache/samza/clus

[GitHub] prateekm commented on issue #934: SAMZA-1935 : Refactor TaskContextImpl to not include access to objects that are only used internally

prateekm commented on issue #934: SAMZA-1935 : Refactor TaskContextImpl to not include access to objects that are only used internally URL: https://github.com/apache/samza/pull/934#issuecomment-467967351 Thanks for the contribution @vwidin. cc @cameronlee314 for review. ---

[GitHub] vwidin opened a new pull request #934: SAMZA-1935 : Refactor TaskContextImpl to not include access to objects that are only used internally

vwidin opened a new pull request #934: SAMZA-1935 : Refactor TaskContextImpl to not include access to objects that are only used internally URL: https://github.com/apache/samza/pull/934 We created a new class called JobContextMetadata which hold the methods `registerObject`, `fetchObject`,