Re: empty operator/stream/module names

2016-08-04 Thread Tushar Gosavi
System generated names can also be problematic. User given name collides with system generated names. we can not generate the name when component is added in the DAG. we will have to wait till all components are added then generate the names. I am -0 on system generated names. Providing a name to o

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408992#comment-15408992 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73648435 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -286,149 +146,119 @@ public AbstractJdbcPollIn

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73648086 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408985#comment-15408985 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73648038 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -286,149 +146,119 @@ public AbstractJdbcPollIn

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408984#comment-15408984 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

Re: empty operator/stream/module names

2016-08-04 Thread Yogi Devendra
1. I am not clear how end user will configure properties for operators with system generated names. 2. If we are going for system generated names we should make sure that names are deterministic and consistent. An operator should get same system generated name for multiple runs. 3

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408980#comment-15408980 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647809 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408979#comment-15408979 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647799 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408976#comment-15408976 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408974#comment-15408974 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #351: APEXMALHAR-2169 Fixed the issue of dynamic pa...

2016-08-04 Thread siyuanh
Github user siyuanh commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/351#discussion_r73647730 --- Diff: contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java --- @@ -188,6 +188,8 @@ @Min(1) private int

[jira] [Commented] (APEXMALHAR-2169) KafkaInputOperator: Dynamic partition is not working in case of ONE_TO_MANY partition strategy

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408975#comment-15408975 ] ASF GitHub Bot commented on APEXMALHAR-2169: Github user siyuanh commente

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647746 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647661 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[jira] [Commented] (APEXMALHAR-2169) KafkaInputOperator: Dynamic partition is not working in case of ONE_TO_MANY partition strategy

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408973#comment-15408973 ] ASF GitHub Bot commented on APEXMALHAR-2169: Github user tushargosavi com

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647479 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -51,22 +64,22 @@ import com.datatorrent.l

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408971#comment-15408971 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408972#comment-15408972 ] ASF GitHub Bot commented on APEXMALHAR-2172: Github user DT-Priyanka comm

[GitHub] apex-malhar pull request #351: APEXMALHAR-2169 Fixed the issue of dynamic pa...

2016-08-04 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/351#discussion_r73647526 --- Diff: contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java --- @@ -188,6 +188,8 @@ @Min(1) private

[GitHub] apex-malhar pull request #358: [review only]APEXMALHAR-2172: Updates to JDBC...

2016-08-04 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/358#discussion_r73647507 --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcPollInputOperator.java --- @@ -80,201 +93,48 @@ * @tags database, sql,

[jira] [Commented] (APEXMALHAR-2169) KafkaInputOperator: Dynamic partition is not working in case of ONE_TO_MANY partition strategy

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408949#comment-15408949 ] ASF GitHub Bot commented on APEXMALHAR-2169: Github user tushargosavi com

[GitHub] apex-malhar pull request #351: APEXMALHAR-2169 Fixed the issue of dynamic pa...

2016-08-04 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/351#discussion_r73646280 --- Diff: contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java --- @@ -628,6 +630,11 @@ else if (newWaitingPartition

[jira] [Commented] (APEXCORE-496) Provide Operator name to StatsListener

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408936#comment-15408936 ] ASF GitHub Bot commented on APEXCORE-496: - Github user tushargosavi commented on

[GitHub] apex-core pull request #361: APEXCORE-496 make operator name available to St...

2016-08-04 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/apex-core/pull/361#discussion_r73645793 --- Diff: api/src/main/java/com/datatorrent/api/StatsListener.java --- @@ -113,6 +113,8 @@ long getLatencyMA(); List getOperato

[jira] [Commented] (APEXCORE-501) Sub Partitioning the parallel partitions

2016-08-04 Thread Yogi Devendra (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408917#comment-15408917 ] Yogi Devendra commented on APEXCORE-501: Link for the discussion: https://lists

[jira] [Created] (APEXCORE-501) Sub Partitioning the parallel partitions

2016-08-04 Thread Yogi Devendra (JIRA)
Yogi Devendra created APEXCORE-501: -- Summary: Sub Partitioning the parallel partitions Key: APEXCORE-501 URL: https://issues.apache.org/jira/browse/APEXCORE-501 Project: Apache Apex Core Iss

[jira] [Commented] (APEXMALHAR-1701) Deduper : create a deduper backed by Managed State

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408904#comment-15408904 ] ASF GitHub Bot commented on APEXMALHAR-1701: Github user asfgit closed th

[GitHub] apex-malhar pull request #335: APEXMALHAR-1701 Deduper with Managed state

2016-08-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: empty operator/stream/module names

2016-08-04 Thread Tushar Gosavi
When we need to change plan dynamically through dtcli, we need name to delete or attach to existing operator/port. I am fine with using system generated name when user do not provide name while adding operator/module/stream. -Tushar. On Thu, Aug 4, 2016 at 10:33 PM, Sanjay Pujare wrote: > I dif

[jira] [Updated] (APEXMALHAR-1701) Deduper : create a deduper backed by Managed State

2016-08-04 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-1701: -- Assignee: Bhupesh Chawda > Deduper : create a deduper backed by Managed Sta

Re: empty operator/stream/module names

2016-08-04 Thread Vlad Rozov
Names are already validated for uniqueness. I believe we still allow names that are not fully compatible with the REST API, but otherwise all unique names are considered to be valid. Vlad On 8/4/16 10:44, Sanjay Pujare wrote: That’s a good point. System generated names can still be made to wo

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408560#comment-15408560 ] ASF GitHub Bot commented on APEXCORE-495: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73610135 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfigPa

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408549#comment-15408549 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73609127 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfig

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408380#comment-15408380 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73589696 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfig

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73586551 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -53,16 +56,16 @@ private static File configFile; -

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408361#comment-15408361 ] ASF GitHub Bot commented on APEXCORE-495: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73584686 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfigPa

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408347#comment-15408347 ] ASF GitHub Bot commented on APEXCORE-495: - Github user sandeshh commented on a d

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408337#comment-15408337 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73583092 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -53,16 +56,16 @@ private static File configFile;

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408332#comment-15408332 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408333#comment-15408333 ] ASF GitHub Bot commented on APEXCORE-495: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73582926 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfigPa

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73582886 --- Diff: engine/src/test/resources/testConfigPackage/testConfigPackageSrc/app/testApp.json --- @@ -0,0 +1 @@ +{"displayName":"testApp","attributes

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408330#comment-15408330 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73582758 --- Diff: engine/src/test/resources/testAppPackage/mydtapp/src/main/resources/com/example/mydtapp/Application/ExtRes.html --- @@ -0,0 +1,21 @@ +

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408328#comment-15408328 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73582496 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -126,10 +130,14 @@ public void testLaunchAppPackagePropertyPrecedence()

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408318#comment-15408318 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73582122 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -103,8 +107,8 @@ public static void finished() public void testL

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408314#comment-15408314 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Amol Kekre
hmm! actually it may be a good debugging tool too. Keep the named checkpoints around. The feature is to keep checkpoints around, which can be done by giving a feature to not delete checkpoints, but then naming them makes it more operational. Send a command from cli->get checkpoint -> know it is the

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73581887 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfig

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408312#comment-15408312 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73581539 --- Diff: engine/src/test/java/com/datatorrent/stram/cli/ApexCliTest.java --- @@ -180,4 +188,40 @@ public void testLaunchAppPackagePrecedenceWithConfig

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408307#comment-15408307 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73581176 --- Diff: engine/src/main/java/com/datatorrent/stram/client/AppPackage.java --- @@ -49,6 +49,8 @@ import net.lingala.zip4j.exception.ZipException;

[GitHub] apex-core pull request #360: APEXCORE-495 supporting apps in config package.

2016-08-04 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-core/pull/360#discussion_r73580877 --- Diff: apex-conf-archetype/src/main/resources/archetype-resources/pom.xml --- @@ -25,6 +25,7 @@ 1.. classpath/* fi

[jira] [Commented] (APEXCORE-495) Enhancing the configuration package to store apps

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408301#comment-15408301 ] ASF GitHub Bot commented on APEXCORE-495: - Github user davidyan74 commented on a

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Venkatesh Kottapalli
+ 1 for the idea. It might be helpful to developers as well when dealing with variety of data in large volumes if this can help them run from the checkpointed state rather than rerunning the application altogether in case of issues. I have seen cases where the application runs for more than 10

Re: empty operator/stream/module names

2016-08-04 Thread Amol Kekre
I am not suggesting no names, AM should allocate system generated names and they will then show up. It should even log it. Thks Amol On Thu, Aug 4, 2016 at 10:44 AM, Sanjay Pujare wrote: > That’s a good point. System generated names can still be made to work for > this use-case but I see the re

Re: empty operator/stream/module names

2016-08-04 Thread Sanjay Pujare
That’s a good point. System generated names can still be made to work for this use-case but I see the reason for having a name. But then another set of questions come up: we need to validate the name for uniqueness within an app, valid syntax etc. May be it’s already being done. On 8/4/16, 10:3

Re: empty operator/stream/module names

2016-08-04 Thread Munagala Ramanath
A couple of minor issues with empty names: It will not be possible to configure such operators from an XML file other than through wildcards. Also, the AM log messages will not be as informative with an empty name, e.g. 2016-08-02 10:04:24,294 INFO com.datatorrent.stram.ResourceRequestHandler: S

Re: empty operator/stream/module names

2016-08-04 Thread Munagala Ramanath
It will not be possible to configure such operators from an XML file other than through wildcards -- but maybe that's OK. Ram On Thu, Aug 4, 2016 at 10:03 AM, Sanjay Pujare wrote: > I differ. For the UI to render a DAG the names are useful, but if the name > is not required by the engine i.e. t

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Amol Kekre
We had an user who wanted roll-back and restart from audit purposes. That time we did not have timed-window. Names checkpoint would have helped a little bit.. Problem statement: Auditors ask for rerun of yesterday's computations for verification. Assume that these computations depend on previous s

Re: empty operator/stream/module names

2016-08-04 Thread Amol Kekre
I agree with Sanjay. Error should be what the engine cannot do without. The lesser work an user can get away with (a choice for a forced option) the better. So if users want they can name these objects, but if they don't it still works. Thks Amol On Thu, Aug 4, 2016 at 10:03 AM, Sanjay Pujare w

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Sanjay Pujare
I agree. A specific use-case will be useful to support this feature. Also the ability to replay from the named checkpoint will be limited because of various factors, isn’t it? On 8/4/16, 9:00 AM, "Pramod Immaneni" wrote: There is a problem here, keeping old checkpoints and recovering from

Re: empty operator/stream/module names

2016-08-04 Thread Sanjay Pujare
I differ. For the UI to render a DAG the names are useful, but if the name is not required by the engine i.e. the engine is able to execute your application fine with empty or null strings as names, is there any reason to make them mandatory? On the other hand, we can come up with a scheme for

Re: empty operator/stream/module names

2016-08-04 Thread Munagala Ramanath
I don't see any reason to allow either. Ram On Thu, Aug 4, 2016 at 8:51 AM, Vlad Rozov wrote: > Currently addOperator/addStream/addModule allows both null and empty > string in the operator/stream/module names. Is there any reason to allow > empty string? Should empty string and null be disallo

Re: empty operator/stream/module names

2016-08-04 Thread Pramod Immaneni
I don't see a reason to have it. On Thu, Aug 4, 2016 at 8:51 AM, Vlad Rozov wrote: > Currently addOperator/addStream/addModule allows both null and empty > string in the operator/stream/module names. Is there any reason to allow > empty string? Should empty string and null be disallowed in those

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Pramod Immaneni
There is a problem here, keeping old checkpoints and recovering from them means preserving the old input data along with the state. This is more than the mechanism of actually creating named checkpoints, it means having the ability for operators to move forward (a.k.a committed and dropping committ

empty operator/stream/module names

2016-08-04 Thread Vlad Rozov
Currently addOperator/addStream/addModule allows both null and empty string in the operator/stream/module names. Is there any reason to allow empty string? Should empty string and null be disallowed in those APIs? Vlad

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Aniruddha Thombare
Sandesh, Agreed. There should be external APIs to acces the feature, if we want to integrate it to 3rd party tools. Thanks, A _ Sent with difficulty, I mean handheld ;) On 4 Aug 2016 8:47 pm, "Sandesh Hegde" wrote: > @Chinmay > We can enhance the existing ch

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Vlad Rozov
+1 for the feature. At first look I am more in favor of reusing existing control tuple. Thank you, Vlad On 8/4/16 08:17, Sandesh Hegde wrote: @Chinmay We can enhance the existing checkpoint tuple but that one is more frequently used than this feature, so why burden Checkpoint tuple with an ex

[jira] [Commented] (APEXCORE-448) Make operator name available in OperatorContext

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407955#comment-15407955 ] ASF GitHub Bot commented on APEXCORE-448: - Github user PramodSSImmaneni commente

[GitHub] apex-core issue #364: APEXCORE-448 Made operator name available in operator ...

2016-08-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the issue: https://github.com/apache/apex-core/pull/364 Yes @chandnisingh please make it volatile or have some other mechanism of passing the information between threads. --- If your project is set up for it, you can reply to this email and h

[jira] [Commented] (APEXCORE-448) Make operator name available in OperatorContext

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407941#comment-15407941 ] ASF GitHub Bot commented on APEXCORE-448: - Github user vrozov commented on the i

[GitHub] apex-core issue #364: APEXCORE-448 Made operator name available in operator ...

2016-08-04 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/apex-core/pull/364 As far as I can see not all changes are correct (please see my comments regarding UnifierDeployInfo and access from two different threads to the operatorName). It will be good to have them fixed an

[jira] [Commented] (APEXCORE-448) Make operator name available in OperatorContext

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407939#comment-15407939 ] ASF GitHub Bot commented on APEXCORE-448: - Github user PramodSSImmaneni commente

[GitHub] apex-core issue #364: APEXCORE-448 Made operator name available in operator ...

2016-08-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the issue: https://github.com/apache/apex-core/pull/364 I second it, please reopen this PR and I will merge it. I think we have spent enough time on the tests and everyone's inputs have been valuable. There will be other opportunities in futur

[jira] [Assigned] (APEXCORE-498) Named Checkpoints - Checkpoint the DAG with a name/tag and start the app from that point

2016-08-04 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh reassigned APEXCORE-498: Assignee: Sandesh > Named Checkpoints - Checkpoint the DAG with a name/tag and start the app from

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Sandesh Hegde
@Chinmay We can enhance the existing checkpoint tuple but that one is more frequently used than this feature, so why burden Checkpoint tuple with an extra field? @Aniruddha It is better to leave the scheduling to the users, they can use any tool that they are already familiar with. On Thu, Aug 4,

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Aniruddha Thombare
+1 On the idea, it would be awesome to have. Question: Can we further develop this brilliant idea into:- Scheduled checkpoints ( To save as dynamically named checkpoint)? This would be on the lines of logrotate / general backup strategies. Thanks, A _ Sent

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Munagala Ramanath
+1 Ram On Thu, Aug 4, 2016 at 12:10 AM, Sandesh Hegde wrote: > Hello Team, > > This thread is to discuss the Named Checkpoint feature for Apex. ( > https://issues.apache.org/jira/browse/APEXCORE-498) > > Named checkpoints allow following workflow, > > 1. Users can trigger a checkpoint and give

[jira] [Commented] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407850#comment-15407850 ] ASF GitHub Bot commented on APEXMALHAR-2176: GitHub user yogidevendra ope

[GitHub] apex-malhar pull request #361: APEXMALHAR-2176 expressionFunctions for Filte...

2016-08-04 Thread yogidevendra
GitHub user yogidevendra opened a pull request: https://github.com/apache/apex-malhar/pull/361 APEXMALHAR-2176 expressionFunctions for FilterOperator 1. added setExpressionFunctionsItem method You can merge this pull request into a Git repository by running: $ git pull https://

[jira] [Comment Edited] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

2016-08-04 Thread Yogi Devendra (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407815#comment-15407815 ] Yogi Devendra edited comment on APEXMALHAR-2176 at 8/4/16 2:10 PM:

[jira] [Commented] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

2016-08-04 Thread Yogi Devendra (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407815#comment-15407815 ] Yogi Devendra commented on APEXMALHAR-2176: --- additional setter method for l

[jira] [Created] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

2016-08-04 Thread Yogi Devendra (JIRA)
Yogi Devendra created APEXMALHAR-2176: - Summary: expressionFunctions for FilterOperator throws IndexOutOfBounds Key: APEXMALHAR-2176 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2176 Proj

[jira] [Created] (APEXMALHAR-2175) POJOUtils is considering java reserved words as fields in the expression

2016-08-04 Thread Yogi Devendra (JIRA)
Yogi Devendra created APEXMALHAR-2175: - Summary: POJOUtils is considering java reserved words as fields in the expression Key: APEXMALHAR-2175 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2175

[jira] [Commented] (APEXCORE-448) Make operator name available in OperatorContext

2016-08-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407454#comment-15407454 ] ASF GitHub Bot commented on APEXCORE-448: - Github user tweise commented on the i

[GitHub] apex-core issue #364: APEXCORE-448 Made operator name available in operator ...

2016-08-04 Thread tweise
Github user tweise commented on the issue: https://github.com/apache/apex-core/pull/364 This pull request attracts a lot of attention... Not to spoil the party, but this looks like a rather minor change and there are many other pull requests that I would consider of much higher import

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Chinmay Kolhatkar
Nice feature. +1 for it. One question, instead of creating a new tuple type, can this be done by modifying current checkpoint tuple? On Thu, Aug 4, 2016 at 2:02 PM, Yogi Devendra wrote: > This will be awesome feature. I can see the usecases for production > scenario as well as developement/tr

  1   2   >