[GitHub] apex-core pull request #439: APEXCORE-595: Don't update committedWindowId wh...

2017-01-03 Thread tushargosavi
GitHub user tushargosavi opened a pull request: https://github.com/apache/apex-core/pull/439 APEXCORE-595: Don't update committedWindowId when all partitions are removed. @tweise please review. You can merge this pull request into a Git repository by running: $ git pull https:

[jira] [Commented] (APEXCORE-595) Master incorrectly updates committedWindowId when all partitions are terminated.

2017-01-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15794689#comment-15794689 ] ASF GitHub Bot commented on APEXCORE-595: - GitHub user tushargosavi opened a pul

[jira] [Commented] (APEXCORE-580) Interface for processing and emitting control tuples

2017-01-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795105#comment-15795105 ] ASF GitHub Bot commented on APEXCORE-580: - GitHub user bhupeshchawda opened a pu

[GitHub] apex-core pull request #440: [REVIEW ONLY] - APEXCORE-580-581 custom control...

2017-01-03 Thread bhupeshchawda
GitHub user bhupeshchawda opened a pull request: https://github.com/apache/apex-core/pull/440 [REVIEW ONLY] - APEXCORE-580-581 custom control tuple support You can merge this pull request into a Git repository by running: $ git pull https://github.com/bhupeshchawda/apex-core

Re: [DISCUSS] Custom Control Tuples Design

2017-01-03 Thread Bhupesh Chawda
Hi All, I have created a review only PR based on the discussion so far. This will also help make the discussion easier and we can continue with the review in parallel. Here is the PR: https://github.com/apache/apex-core/pull/440 Please help review this. I am still working on documentation and te

[GitHub] apex-malhar pull request #527: APEXMALHAR-2374 Calling scan recursively if t...

2017-01-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/527 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (APEXMALHAR-2374) Recursive support for AbstractFileInputOperator

2017-01-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795127#comment-15795127 ] ASF GitHub Bot commented on APEXMALHAR-2374: Github user asfgit closed th

[jira] [Resolved] (APEXMALHAR-2374) Recursive support for AbstractFileInputOperator

2017-01-03 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda resolved APEXMALHAR-2374. Resolution: Fixed Fix Version/s: 3.7.0 > Recursive support for AbstractF

[GitHub] apex-core pull request #431: Add Apache 2.0 license to META-INF/services fil...

2017-01-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: [DISCUSS] Custom Control Tuples Design

2017-01-03 Thread David Yan
The annotation should be true by default. If an operator does not care about the control tuples, it should propagate them because the downstream might care about it. For example, let's say the original DAG looks like: A->B And A emits control tuples that B cares about, and of course along with ot

[jira] [Updated] (APEXCORE-598) Embedded mode execution does not use APPLICATION_PATH for checkpointing

2017-01-03 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXCORE-598: -- Affects Version/s: 3.5.0 > Embedded mode execution does not use APPLICATION_PATH for checkpoin

[jira] [Created] (APEXCORE-598) Embedded mode execution does not use APPLICATION_PATH for checkpointing

2017-01-03 Thread Thomas Weise (JIRA)
Thomas Weise created APEXCORE-598: - Summary: Embedded mode execution does not use APPLICATION_PATH for checkpointing Key: APEXCORE-598 URL: https://issues.apache.org/jira/browse/APEXCORE-598 Project:

[GitHub] apex-core pull request #441: APEXCORE-598 Write checkpoints to APPLICATION_P...

2017-01-03 Thread tweise
GitHub user tweise opened a pull request: https://github.com/apache/apex-core/pull/441 APEXCORE-598 Write checkpoints to APPLICATION_PATH in embedded execution mode. @davidyan74 or @vrozov please review. You can merge this pull request into a Git repository by running: $ git p

[jira] [Commented] (APEXCORE-598) Embedded mode execution does not use APPLICATION_PATH for checkpointing

2017-01-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795674#comment-15795674 ] ASF GitHub Bot commented on APEXCORE-598: - GitHub user tweise opened a pull requ

Re: [DISCUSS] Custom Control Tuples Design

2017-01-03 Thread Bhupesh Chawda
Yes David, that is correct. The annotation is true by default. However, it may happen that in the course of processing, we may want to stop the propagation of control tuples to the downstream operators. In this case, we should have an option to block the control tuples propagation to downstream op

[GitHub] apex-malhar pull request #500: APEXMALHAR-2343 #resolve #comment Count Accum...

2017-01-03 Thread brightchen
Github user brightchen closed the pull request at: https://github.com/apache/apex-malhar/pull/500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[jira] [Commented] (APEXMALHAR-2343) Count Accumulation should only increase one for each tuple

2017-01-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796610#comment-15796610 ] ASF GitHub Bot commented on APEXMALHAR-2343: Github user brightchen close

Re: [DISCUSS] Custom Control Tuples Design

2017-01-03 Thread Pramod Immaneni
2 sounds good. Have you thought about what the method would look like. On Sat, Dec 31, 2016 at 8:29 PM, Bhupesh Chawda wrote: > Yes, that makes sense. > We have following options: > 1. Make the annotation false by default and force the user to forward the > control tuples explicitly. > 2. Annota

Re: [DISCUSS] Custom Control Tuples Design

2017-01-03 Thread Bhupesh Chawda
Hi Pramod, I was thinking of a method setPropagateControlTuples(boolean propagate) on the output port of the operator. The operator could disable this in the code at any point of time. Note however that this is to block the propagation of control tuples from upstream. Any control tuples emitted ex