[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265102#comment-15265102 ] Thomas Weise commented on APEXMALHAR-2068: -- Will leave the JIRA open for the

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265100#comment-15265100 ] ASF GitHub Bot commented on APEXMALHAR-2068: Github user asfgit closed th

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2068 made noop defa...

2016-04-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/259 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: Java Stream API Pull Request

2016-04-29 Thread Siyuan Hua
Sorry, here is the link for the pull request https://github.com/apache/incubator-apex-malhar/pull/261 On Fri, Apr 29, 2016 at 5:43 PM, Siyuan Hua wrote: > Hi Community, > > Happy Friday! > I just sent out an initial pull request for Java High Level Stream API. > This is the very first attempt

Java Stream API Pull Request

2016-04-29 Thread Siyuan Hua
Hi Community, Happy Friday! I just sent out an initial pull request for Java High Level Stream API. This is the very first attempt to bring the functional paradigm into Apex programming model. We will keep working on this for months, if you are interested, please take time to have a look and make

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264981#comment-15264981 ] ASF GitHub Bot commented on APEXMALHAR-2068: Github user chandnisingh com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2068 made noop defa...

2016-04-29 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/259#discussion_r61659396 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java --- @@ -96,7 +96,7 @@ public FileSplitterInput()

[jira] [Commented] (APEXCORE-447) Document: update AutoMetrics with AppDataTracker link

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264902#comment-15264902 ] ASF GitHub Bot commented on APEXCORE-447: - Github user asfgit closed the pull re

[GitHub] incubator-apex-core pull request: APEXCORE-447 updated app data tr...

2016-04-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-core/pull/320 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[jira] [Resolved] (APEXCORE-447) Document: update AutoMetrics with AppDataTracker link

2016-04-29 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh resolved APEXCORE-447. Resolution: Fixed > Document: update AutoMetrics with AppDataTracker link > --

[jira] [Commented] (APEXCORE-447) Document: update AutoMetrics with AppDataTracker link

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264895#comment-15264895 ] ASF GitHub Bot commented on APEXCORE-447: - Github user sashadt commented on the

[jira] [Updated] (APEXCORE-447) Document: update AutoMetrics with AppDataTracker link

2016-04-29 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXCORE-447: --- Fix Version/s: 3.4.0 > Document: update AutoMetrics with AppDataTracker link > -

[GitHub] incubator-apex-core pull request: APEXCORE-447 updated app data tr...

2016-04-29 Thread sashadt
Github user sashadt commented on the pull request: https://github.com/apache/incubator-apex-core/pull/320#issuecomment-215898480 Looks good. Merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[jira] [Created] (APEXCORE-448) Make operator name available in OperatorContext

2016-04-29 Thread Chandni Singh (JIRA)
Chandni Singh created APEXCORE-448: -- Summary: Make operator name available in OperatorContext Key: APEXCORE-448 URL: https://issues.apache.org/jira/browse/APEXCORE-448 Project: Apache Apex Core

[jira] [Commented] (APEXCORE-107) Support adding module to application using property file API.

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264575#comment-15264575 ] ASF GitHub Bot commented on APEXCORE-107: - Github user tushargosavi commented on

[jira] [Commented] (APEXCORE-107) Support adding module to application using property file API.

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264574#comment-15264574 ] ASF GitHub Bot commented on APEXCORE-107: - Github user tushargosavi commented on

[jira] [Commented] (APEXCORE-107) Support adding module to application using property file API.

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264573#comment-15264573 ] ASF GitHub Bot commented on APEXCORE-107: - Github user tushargosavi commented on

[GitHub] incubator-apex-core pull request: APEXCORE-107 Support for adding ...

2016-04-29 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/313#discussion_r61630974 --- Diff: api/src/main/java/com/datatorrent/api/DagNode.java --- @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] incubator-apex-core pull request: APEXCORE-107 Support for adding ...

2016-04-29 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/313#discussion_r61630938 --- Diff: engine/src/test/java/com/datatorrent/stram/plan/logical/module/TestModuleExpansion.java --- @@ -184,7 +190,7 @@ public void setPo

[GitHub] incubator-apex-core pull request: APEXCORE-107 Support for adding ...

2016-04-29 Thread tushargosavi
Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/313#discussion_r61630963 --- Diff: engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlanConfiguration.java --- @@ -2187,6 +2190,27 @@ public void popu

[GitHub] incubator-apex-malhar pull request: Stream API

2016-04-29 Thread siyuanh
GitHub user siyuanh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/261 Stream API You can merge this pull request into a Git repository by running: $ git pull https://github.com/siyuanh/incubator-apex-malhar stream Alternatively you can review an

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2068 made noop defa...

2016-04-29 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/259#discussion_r61627655 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java --- @@ -96,7 +96,7 @@ public FileSplitterInput()

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264530#comment-15264530 ] ASF GitHub Bot commented on APEXMALHAR-2068: Github user PramodSSImmaneni

[jira] [Commented] (APEXMALHAR-1701) Deduper : create a deduper backed by Managed State

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264527#comment-15264527 ] ASF GitHub Bot commented on APEXMALHAR-1701: GitHub user chandnisingh ope

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1701 An abstract de...

2016-04-29 Thread chandnisingh
GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/260 APEXMALHAR-1701 An abstract deducer implementation [Review Only] This is an abstract deduplicator that uses managed state. @bhupeshchawda if you like you can use this base imp

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264499#comment-15264499 ] ASF GitHub Bot commented on APEXMALHAR-2068: GitHub user chandnisingh ope

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2068 made noop defa...

2016-04-29 Thread chandnisingh
GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/259 APEXMALHAR-2068 made noop default in FileSplitter @tweise @vrozov made noop default in FileSplitterInput You can merge this pull request into a Git repository by running: $

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264429#comment-15264429 ] Chandni Singh commented on APEXMALHAR-2068: --- Ok > FileSplitter- having mul

Re: Best practises to specify the Application

2016-04-29 Thread Munagala Ramanath
In the pom.xml file of the 3.3 archetype generated project I'm already seeing: 1.7 1.7 for the maven-compiler-plugin configuration Ram On Fri, Apr 29, 2016 at 10:37 AM, David Yan wrote: > [switching to dev list] > > It appears Ananth is using JDK 1.8 to compile the apa and

Re: Best practises to specify the Application

2016-04-29 Thread David Yan
Ah, never mind. -target 1.7 is already there. https://github.com/apache/incubator-apex-core/blob/master/apex-app-archetype/src/main/resources/archetype-resources/pom.xml It boils down to the user not using the archetype. David On Fri, Apr 29, 2016 at 10:47 AM, Munagala Ramanath wrote: > A bunc

Re: Best practises to specify the Application

2016-04-29 Thread Munagala Ramanath
A bunch of us are using JDK 1.8 to compile apps and deploying to our cluster and/or sandbox and not seeing this issue; how come ? Ram On Fri, Apr 29, 2016 at 10:37 AM, David Yan wrote: > [switching to dev list] > > It appears Ananth is using JDK 1.8 to compile the apa and our distributed > bina

[jira] [Commented] (APEXCORE-107) Support adding module to application using property file API.

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264405#comment-15264405 ] ASF GitHub Bot commented on APEXCORE-107: - Github user sandeshh commented on a d

[GitHub] incubator-apex-core pull request: APEXCORE-107 Support for adding ...

2016-04-29 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/313#discussion_r61617386 --- Diff: api/src/main/java/com/datatorrent/api/DagNode.java --- @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Foundation (AS

[jira] [Commented] (APEXCORE-107) Support adding module to application using property file API.

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264400#comment-15264400 ] ASF GitHub Bot commented on APEXCORE-107: - Github user PramodSSImmaneni commente

[GitHub] incubator-apex-core pull request: APEXCORE-107 Support for adding ...

2016-04-29 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/313#discussion_r61617019 --- Diff: api/src/main/java/com/datatorrent/api/DagNode.java --- @@ -0,0 +1,24 @@ +/** + * Licensed to the Apache Software Founda

Fwd: Best practises to specify the Application

2016-04-29 Thread David Yan
[switching to dev list] It appears Ananth is using JDK 1.8 to compile the apa and our distributed binary, compiled with 1.7 by us, cannot read class files compiled with 1.8. I think we may need to add target=1.7 in the compiler plugin config in the pom.xml file of apex-app-archetype. Thoughts?

[jira] [Commented] (APEXMALHAR-2023) Adding Enrichment Operator to Malhar

2016-04-29 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264366#comment-15264366 ] ASF GitHub Bot commented on APEXMALHAR-2023: Github user chinmaykolhatkar

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2023 Enricher

2016-04-29 Thread chinmaykolhatkar
Github user chinmaykolhatkar commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/235#discussion_r61614306 --- Diff: contrib/src/main/java/com/datatorrent/contrib/enrich/FSLoader.java --- @@ -0,0 +1,181 @@ +/** + * Licensed to the A

[jira] [Commented] (APEXMALHAR-2068) FileSplitter- having multiple file splitters in an application results in incorrect behavior

2016-04-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264128#comment-15264128 ] Thomas Weise commented on APEXMALHAR-2068: -- Can we in the the interim make t

Re: Reading large HDFS files record by record

2016-04-29 Thread Yogi Devendra
Yes. Single file read in parallel will be supported similar to FileSplitter+BlockReader combination. ~ Yogi On 29 April 2016 at 15:58, Sandeep Deshmukh wrote: > +1 > > Will this support reading a single file in parallel? > On 29-Apr-2016 3:27 pm, "Mohit Jotwani" wrote: > > > +1 > > > > Regards

Re: Reading large HDFS files record by record

2016-04-29 Thread Sandeep Deshmukh
+1 Will this support reading a single file in parallel? On 29-Apr-2016 3:27 pm, "Mohit Jotwani" wrote: > +1 > > Regards, > Mohit > > On Thu, Apr 28, 2016 at 4:29 PM, Yogi Devendra < > devendra.vyavah...@gmail.com > > wrote: > > > Hi, > > > > My usecase involves reading from HDFS and emit each re

Re: Reading large HDFS files record by record

2016-04-29 Thread Mohit Jotwani
+1 Regards, Mohit On Thu, Apr 28, 2016 at 4:29 PM, Yogi Devendra wrote: > Hi, > > My usecase involves reading from HDFS and emit each record as a separate > tuple. Record can be either fixed length record or separator based record > (such as newline). Expected output is byte[] for each record.

Re: NFS Input Module

2016-04-29 Thread Mohit Jotwani
+1 Regards, Mohit On Fri, Apr 29, 2016 at 2:09 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Hi All, > > I am proposing NFS Input Module. Use case is to read large files from NFS > in parallel. > > Design of NFS input module: > >There is a common interface "FSInputModule" in

NFS Input Module

2016-04-29 Thread Chaitanya Chebolu
Hi All, I am proposing NFS Input Module. Use case is to read large files from NFS in parallel. Design of NFS input module: There is a common interface "FSInputModule" in Malhar for the input Modules. NFS input Module extends from FSInputModule and can be achieved by using FSFileSplitter an