[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2013 : HDFS output ...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/216#discussion_r56624309 --- Diff: apps/filecopy/src/main/java/com/datatorrent/apps/copy/HDFSFileCopySampleApp.java --- @@ -0,0 +1,50 @@ +/** + * Licensed

[GitHub] incubator-apex-malhar pull request: Apexmalhar 2013 hdfs file copy...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/213#discussion_r56618129 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileSplitter.java --- @@ -365,6 +365,16 @@ public FileMetadata(@NotNull

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2010 Add Tranform o...

2016-03-19 Thread sandeepdeshmukh
Github user sandeepdeshmukh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/209#discussion_r56652223 --- Diff: library/src/main/java/com/datatorrent/lib/transform/TransformOperator.java --- @@ -0,0 +1,172 @@ +/** + * Licensed t

[jira] [Updated] (APEXCORE-388) Links to 3rd party pages should follow guidelines defined at http://www.apache.org/foundation/marks/linking

2016-03-19 Thread Sasha Parfenov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasha Parfenov updated APEXCORE-388: Assignee: Sasha Parfenov > Links to 3rd party pages should follow guidelines defined at >

[jira] [Commented] (APEXCORE-371) Update ASF and incubator logos on apex web site

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198564#comment-15198564 ] ASF GitHub Bot commented on APEXCORE-371: - GitHub user sashadt opened a pull req

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201288#comment-15201288 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2013 : HDFS output ...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/216#discussion_r56624529 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/BlockWriter.java --- @@ -0,0 +1,210 @@ +/** + * Licensed to the Apache S

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2013 : HDFS output ...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/216#discussion_r56625652 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FileMerger.java --- @@ -0,0 +1,161 @@ +/** + * Licensed to the Apache So

[jira] [Commented] (APEXMALHAR-2015) Projection Operator

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201417#comment-15201417 ] ASF GitHub Bot commented on APEXMALHAR-2015: Github user yogidevendra com

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203087#comment-15203087 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user chandnisingh com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56759046 --- Diff: library/src/main/java/com/datatorrent/lib/state/managed/Bucket.java --- @@ -0,0 +1,534 @@ +/** + * Licensed to the Apac

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203083#comment-15203083 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user ilooner commente

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread ilooner
Github user ilooner commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56759001 --- Diff: library/src/main/java/com/datatorrent/lib/state/managed/Bucket.java --- @@ -0,0 +1,534 @@ +/** + * Licensed to the Apache So

[jira] [Created] (APEXCORE-398) Ack may not be delivered from buffer server to it's client

2016-03-19 Thread Vlad Rozov (JIRA)
Vlad Rozov created APEXCORE-398: --- Summary: Ack may not be delivered from buffer server to it's client Key: APEXCORE-398 URL: https://issues.apache.org/jira/browse/APEXCORE-398 Project: Apache Apex Core

[jira] [Commented] (APEXCORE-391) AsyncFSStorageAgent creates tmp directory unnecessarily

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198424#comment-15198424 ] ASF GitHub Bot commented on APEXCORE-391: - Github user asfgit closed the pull re

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread amberarrow
Github user amberarrow commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56718885 --- Diff: library/src/main/java/com/datatorrent/lib/state/BucketedState.java --- @@ -0,0 +1,60 @@ +/** + * Licensed to the Apache S

[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...

2016-03-19 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/incubator-apex-site/pull/19#discussion_r56676627 --- Diff: src/md/compatibility.md --- @@ -0,0 +1,112 @@ +#Apache Apex Compatibility + +##Purpose + +This document captures the c

[GitHub] incubator-apex-site pull request: APEXCORE-321 bylaws draft

2016-03-19 Thread alanfgates
Github user alanfgates commented on a diff in the pull request: https://github.com/apache/incubator-apex-site/pull/22#discussion_r56566369 --- Diff: src/md/bylaws.md --- @@ -0,0 +1,381 @@ +#*** draft subject to top level project approval *** +#Apache Apex Project Bylaws

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203071#comment-15203071 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user chandnisingh com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56758813 --- Diff: library/src/main/java/com/datatorrent/lib/state/managed/Bucket.java --- @@ -0,0 +1,534 @@ +/** + * Licensed to the Apac

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202110#comment-15202110 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user amberarrow comme

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203070#comment-15203070 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user ilooner commente

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread ilooner
Github user ilooner commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56758801 --- Diff: library/src/main/java/com/datatorrent/lib/state/managed/Bucket.java --- @@ -0,0 +1,534 @@ +/** + * Licensed to the Apache So

[GitHub] incubator-apex-site pull request: APEXCORE-320 add who-we-are page

2016-03-19 Thread tweise
GitHub user tweise opened a pull request: https://github.com/apache/incubator-apex-site/pull/25 APEXCORE-320 add who-we-are page You can merge this pull request into a Git repository by running: $ git pull https://github.com/tweise/incubator-apex-site APEXCORE-320 Alternative

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2010 Add Tranform o...

2016-03-19 Thread sandeepdeshmukh
Github user sandeepdeshmukh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/209#discussion_r56648509 --- Diff: library/src/main/java/com/datatorrent/lib/expression/JavaExpressionParser.java --- @@ -0,0 +1,179 @@ +/** + * Licen

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201289#comment-15201289 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2013 : HDFS output ...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/216#discussion_r56628702 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileCopyModule.java --- @@ -0,0 +1,123 @@ +/** + * Licensed to the A

[GitHub] incubator-apex-core pull request: APEXCORE-383 - Time to sleep whi...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-core/pull/270 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[jira] [Commented] (APEXMALHAR-2010) Add transform operator

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201437#comment-15201437 ] ASF GitHub Bot commented on APEXMALHAR-2010: Github user sandeepdeshmukh

[jira] [Commented] (APEXCORE-321) Document voting rules

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200941#comment-15200941 ] ASF GitHub Bot commented on APEXCORE-321: - Github user justinmclean commented on

Available Resource types

2016-03-19 Thread Ronnie Navarro
ronnie

RE: Stack overflow errors when launching job

2016-03-19 Thread Ganelin, Ilya
I don't think I have any time really to connect to the container. The application launches and crashes almost immediately. Total runtime is 50 seconds. Sent with Good (www.good.com) From: Munagala Ramanath Sent: Saturday, March 19, 2016 5:39:11 PM To: dev@apex

[jira] [Commented] (APEXCORE-388) Links to 3rd party pages should follow guidelines defined at http://www.apache.org/foundation/marks/linking

2016-03-19 Thread Sasha Parfenov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198053#comment-15198053 ] Sasha Parfenov commented on APEXCORE-388: - Replaced all the links across the sit

[jira] [Moved] (APEXMALHAR-2021) Add property to AbstractFileInputOperator to trim processedFiles and ignoredFiles

2016-03-19 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXCORE-343 to APEXMALHAR-2021: --- Affects Version/s: (was: 3.3.0)

[jira] [Commented] (APEXCORE-293) Add core and malhar documentation to project web site

2016-03-19 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201758#comment-15201758 ] Thomas Weise commented on APEXCORE-293: --- [~sasha] We should review the need for ea

Re: Setting list and map properties via properties.xml

2016-03-19 Thread Munagala Ramanath
The list initialization only works when I prepopulate the list with enough elements to accomodate all the items from the property file; otherwise I get a failure: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 Ram On Thu, Mar 17, 2016 at 11:14 AM, Chinmay Kolhatkar wrote: > @Ram, O

Re: Stack overflow errors when launching job

2016-03-19 Thread Ganelin, Ilya
I’ve updated the gist with a more complete example, and updated the associated JIRA that I’ve created. https://issues.apache.org/jira/browse/APEXCORE-392 On 3/17/16, 4:33 AM, "Tushar Gosavi" wrote: >Hi, > >I created a sample application with operators from the given link. just a >simple in

[jira] [Commented] (APEXCORE-393) Reset failure count when consecutive failed node is removed from blacklist

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199898#comment-15199898 ] ASF GitHub Bot commented on APEXCORE-393: - Github user tweise commented on a dif

[GitHub] incubator-apex-malhar pull request: Apexmalhar 2013 hdfs file copy...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/213#discussion_r56499282 --- Diff: apps/filecopy/pom.xml --- @@ -0,0 +1,106 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/20

[jira] [Commented] (APEXCORE-391) AsyncFSStorageAgent creates tmp directory unnecessarily

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198385#comment-15198385 ] ASF GitHub Bot commented on APEXCORE-391: - Github user davidyan74 closed the pul

[jira] [Commented] (APEXCORE-395) Unable to launch Sandbox using Oracle VMBox

2016-03-19 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198823#comment-15198823 ] Thomas Weise commented on APEXCORE-395: --- This is a problem with the DT sandbox and

[jira] [Commented] (APEXMALHAR-1957) Improve HBasePOJOInputOperator with support for threaded read

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199196#comment-15199196 ] ASF GitHub Bot commented on APEXMALHAR-1957: GitHub user bhupeshchawda op

[jira] [Commented] (APEXCORE-393) Reset failure count when consecutive failed node is removed from blacklist

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199124#comment-15199124 ] ASF GitHub Bot commented on APEXCORE-393: - GitHub user ishark opened a pull requ

[jira] [Resolved] (APEXCORE-246) Add capability in the engine to let the operator know about the next checkpoint window

2016-03-19 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pramod Immaneni resolved APEXCORE-246. -- Resolution: Fixed > Add capability in the engine to let the operator know about the ne

[jira] [Assigned] (APEXMALHAR-1779) H2O integration using POJO

2016-03-19 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pramod Immaneni reassigned APEXMALHAR-1779: --- Assignee: Sandesh (was: Pramod Immaneni) Sandesh this is an old JIRA to

[jira] [Resolved] (APEXMALHAR-1858) Provide a ready to use operator that outputs the lines from a file

2016-03-19 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pramod Immaneni resolved APEXMALHAR-1858. - Resolution: Fixed This was done a while back > Provide a ready to use operat

[GitHub] incubator-apex-core pull request: APEXCORE-391 #resolve delay the ...

2016-03-19 Thread davidyan74
GitHub user davidyan74 reopened a pull request: https://github.com/apache/incubator-apex-core/pull/273 APEXCORE-391 #resolve delay the creation of tmp directory until save is called @tweise please cherry pick to release-3.3 if appropriate You can merge this pull request into a Git

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2015: Projection Op...

2016-03-19 Thread yogidevendra
Github user yogidevendra commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/217#discussion_r56648250 --- Diff: library/src/test/java/com/datatorrent/lib/projection/ActivateTest.java --- @@ -0,0 +1,147 @@ +/** + * Licensed to the A

[jira] [Closed] (APEXMALHAR-1833) Convert MachineData demo to support Query Operator

2016-03-19 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munagala V. Ramanath closed APEXMALHAR-1833. Resolution: Fixed This doesn't belong in APEXMALHAR and is also obsolet

Re: S3 Input Module

2016-03-19 Thread Chaitanya Chebolu
Yes Yogi. For Parallel read, we cannot support Hadoop versions below 2.7 without copying. I suggested this approach. On Fri, Mar 18, 2016 at 6:26 PM, Yogi Devendra wrote: > Chaitanya, > > Do you mean to say that we cannot support hadoop versions below 2.7 without > copying few files from Hadoop

[GitHub] incubator-apex-site pull request: APEXCORE-321 bylaws draft

2016-03-19 Thread justinmclean
Github user justinmclean commented on a diff in the pull request: https://github.com/apache/incubator-apex-site/pull/22#discussion_r56610060 --- Diff: src/md/bylaws.md --- @@ -0,0 +1,381 @@ +#*** draft subject to top level project approval *** +#Apache Apex Project Bylaws

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2010 Add Tranform o...

2016-03-19 Thread sandeepdeshmukh
Github user sandeepdeshmukh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/209#discussion_r56652159 --- Diff: library/src/main/java/com/datatorrent/lib/transform/TransformOperator.java --- @@ -0,0 +1,172 @@ +/** + * Licensed t

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2010 Add Tranform o...

2016-03-19 Thread sandeepdeshmukh
Github user sandeepdeshmukh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/209#discussion_r56651325 --- Diff: library/src/test/java/com/datatorrent/lib/util/PojoUtilsTest.java --- @@ -489,4 +483,59 @@ public void testWrongSetterArgume

[jira] [Resolved] (APEXCORE-383) Time to sleep while reservoirs are full should increase from 0 to a configurable max value

2016-03-19 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXCORE-383. --- Resolution: Fixed Fix Version/s: 3.4.0 > Time to sleep while reservoirs are full shou

Re: Stack overflow errors when launching job

2016-03-19 Thread Munagala Ramanath
Ilya, Your stack trace is cut off near the bottom, so we don't know where the call sequence originated. Can you add -XXMaxJavaStackTraceDepth=-1 to the JVM options of all your containers and upload the full stack trace of the failure ? Ram On Sat, Mar 19, 2016 at 2:07 PM, Ganelin, Ilya wrote:

[jira] [Commented] (APEXCORE-393) Reset failure count when consecutive failed node is removed from blacklist

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199902#comment-15199902 ] ASF GitHub Bot commented on APEXCORE-393: - Github user tweise commented on a dif

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2015: Projection Op...

2016-03-19 Thread yogidevendra
Github user yogidevendra commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/217#discussion_r56650591 --- Diff: library/src/test/java/com/datatorrent/lib/projection/ProjectionTest.java --- @@ -0,0 +1,264 @@ +/** + * Licensed to the

Re: [GitHub] incubator-apex-site pull request: APEXCORE-321 bylaws draft

2016-03-19 Thread Justin Mclean
Hi, > GitHub user tweise opened a pull request: > >https://github.com/apache/incubator-apex-site/pull/22 > >APEXCORE-321 bylaws draft > >This is the draft to address remaining items for graduation and maturity > model. It is derived from Apache Tez. Please review thoroughly. Just

Re: S3 Input Module

2016-03-19 Thread Priyanka Gugale
It's a good idea to extract out common code in parent class. +1 for this feature. -Priyanka On Thu, Mar 17, 2016 at 1:57 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Dear Community, > > I am proposing S3 Input Module. Primary functionality of this module is > to parallel read f

Re: Reg. files handled soft limit set in the application

2016-03-19 Thread Sandeep Deshmukh
There is no limit as such from Apex but whatever applies to Hadoop will still be applicable here. So, you may have to do some tweaks in Hadoop. https://wiki.apache.org/hadoop/TooManyOpenFiles http://askubuntu.com/questions/162345/how-to-increase-open-file-limits-nofile-and-epoll-in-10-04 Regards

Re: S3 Input Module

2016-03-19 Thread Chaitanya Chebolu
Hi Sandeep, I am extending from HDFS input module. I am supporting the same features for S3, which are supported by HDFS input module. Please find my comments in-line. On Fri, Mar 18, 2016 at 12:07 PM, Sandeep Deshmukh wrote: > Hi Chaitanya, > > I have a query on parallel reading via S3. W

[GitHub] incubator-apex-malhar pull request: Apexmalhar 2013 hdfs file copy...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/213#discussion_r56500687 --- Diff: apps/filecopy/src/main/java/com/datatorrent/modules/app/HDFSFileCopySampleApp.java --- @@ -0,0 +1,50 @@ +/** + * Licens

[jira] [Updated] (APEXCORE-284) Create a docs folder in core repository

2016-03-19 Thread Sasha Parfenov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasha Parfenov updated APEXCORE-284: Assignee: Sasha Parfenov (was: Andy Perlitch) > Create a docs folder in core repository >

[jira] [Created] (APEXMALHAR-2020) HBase Row key type should be configurable

2016-03-19 Thread Bhupesh Chawda (JIRA)
Bhupesh Chawda created APEXMALHAR-2020: -- Summary: HBase Row key type should be configurable Key: APEXMALHAR-2020 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2020 Project: Apache Apex Mal

Re: [GitHub] incubator-apex-site pull request: APEXCORE-321 bylaws draft

2016-03-19 Thread Justin Mclean
Hi, JFYI It's more common to call bylaws guidelines theses days. > A committer will be considered > “emeritus/inactive” by not contributing in any form to the > project for > over 1 year. This is a little unusual and you may want to discuss this further. One of the principals of t

Re: Stack overflow errors when launching job

2016-03-19 Thread Munagala Ramanath
One, somewhat unorthodox method is described here: http://stackoverflow.com/questions/23632653/generate-java-heap-dump-on-uncaught-exception Ram On Sat, Mar 19, 2016 at 2:07 PM, Ganelin, Ilya wrote: > How would we go about getting a heap dump? > > > > Sent with Good (www.good.com) > __

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2008: Create HDFS F...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/207 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Created] (APEXMALHAR-2017) Use pre checkpoint notification to optimize operator IO

2016-03-19 Thread Pramod Immaneni (JIRA)
Pramod Immaneni created APEXMALHAR-2017: --- Summary: Use pre checkpoint notification to optimize operator IO Key: APEXMALHAR-2017 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2017 Project:

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201189#comment-15201189 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

Re: Stack overflow errors when launching job

2016-03-19 Thread Munagala Ramanath
There is some info here, near the end of the page: http://docs.datatorrent.com/troubleshooting/ under the heading "How do I get a heap dump when a container gets an OutOfMemoryError ?" However since you're blowing the stack, you may need to manually run jmap on the running container which may be

[GitHub] incubator-apex-core pull request: APEXCORE-380 - Idle time sleep t...

2016-03-19 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/271#discussion_r56447191 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/GenericNode.java --- @@ -608,9 +610,9 @@ else if (tracker.ports[trackerIndex] == null)

[jira] [Comment Edited] (APEXCORE-392) Stack Overflow when launching jobs

2016-03-19 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200660#comment-15200660 ] Ilya Ganelin edited comment on APEXCORE-392 at 3/18/16 12:14 AM: -

[jira] [Commented] (APEXMALHAR-2010) Add transform operator

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201419#comment-15201419 ] ASF GitHub Bot commented on APEXMALHAR-2010: Github user sandeepdeshmukh

RE: Stack overflow errors when launching job

2016-03-19 Thread Ganelin, Ilya
How would we go about getting a heap dump? Sent with Good (www.good.com) From: Yogi Devendra Sent: Saturday, March 19, 2016 12:19:26 AM To: dev@apex.incubator.apache.org Subject: Re: Stack overflow errors when launching job Stack trace in the gist shows some sy

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread ilooner
Github user ilooner commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56754934 --- Diff: library/src/main/java/com/datatorrent/lib/state/TimeSlicedBucketedState.java --- @@ -0,0 +1,98 @@ +/** + * Licensed to the A

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202960#comment-15202960 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user ilooner commente

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202959#comment-15202959 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user chandnisingh com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56754928 --- Diff: library/src/main/java/com/datatorrent/lib/state/TimeSlicedBucketedState.java --- @@ -0,0 +1,98 @@ +/** + * Licensed to

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56754895 --- Diff: library/src/main/java/com/datatorrent/lib/state/managed/ManagedTimeUnifiedStateImpl.java --- @@ -0,0 +1,238 @@ +/** +

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202958#comment-15202958 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user chandnisingh com

[jira] [Commented] (APEXMALHAR-1897) Create ManagedState

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202956#comment-15202956 ] ASF GitHub Bot commented on APEXMALHAR-1897: Github user ilooner commente

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1897 added managed ...

2016-03-19 Thread ilooner
Github user ilooner commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/145#discussion_r56754879 --- Diff: library/src/main/java/com/datatorrent/lib/state/TimeSlicedBucketedState.java --- @@ -0,0 +1,98 @@ +/** + * Licensed to the A

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201182#comment-15201182 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2015: Projection Op...

2016-03-19 Thread yogidevendra
Github user yogidevendra commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/217#discussion_r56641705 --- Diff: library/src/main/java/com/datatorrent/lib/projection/ProjectionOperator.java --- @@ -0,0 +1,303 @@ +/** + * Licensed to

Re: Bandwidth control for Input operators in Apex

2016-03-19 Thread Priyanka Gugale
Hi, Thanks for inputs Sandeep, would take care of those points. Here is high level design we are considering, We would have following components: *1.* *BandwidthManager* This keeps track of current bandwidth usage of system and takes decision if requested data bandwidth can be used right away or

Re: S3 Input Module

2016-03-19 Thread Sandeep Deshmukh
Hi Chaitanya, I have a query on parallel reading via S3. Will you be supporting 1. Reading one file in parallel ( say 4 block readers reading the same file 2. Reading multiple files in parallel but a file is always read serially. So different block reader instances read different file

[jira] [Commented] (APEXMALHAR-2015) Projection Operator

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201334#comment-15201334 ] ASF GitHub Bot commented on APEXMALHAR-2015: Github user yogidevendra com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2011-2012 Avro to P...

2016-03-19 Thread chinmaykolhatkar
Github user chinmaykolhatkar commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/211#discussion_r56624094 --- Diff: contrib/src/main/java/com/datatorrent/contrib/avro/AvroRecordHelper.java --- @@ -114,6 +107,7 @@ private static Object con

[GitHub] incubator-apex-malhar pull request: MLHR 1957 hbase input

2016-03-19 Thread bhupeshchawda
Github user bhupeshchawda closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/155 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] incubator-apex-core pull request: Disable blacklisting of failed n...

2016-03-19 Thread ishark
GitHub user ishark opened a pull request: https://github.com/apache/incubator-apex-core/pull/275 Disable blacklisting of failed nodes by default Tested that nodes are not getting blacklisted after 3 failures. You can merge this pull request into a Git repository by running: $ g

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2013 : HDFS output ...

2016-03-19 Thread DT-Priyanka
Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/216#discussion_r56628057 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileCopyModule.java --- @@ -0,0 +1,123 @@ +/** + * Licensed to the A

[GitHub] incubator-apex-malhar pull request: Add Idempotent support the new...

2016-03-19 Thread siyuanh
GitHub user siyuanh reopened a pull request: https://github.com/apache/incubator-apex-malhar/pull/205 Add Idempotent support the new kafka input operator @chaithu14 @tweise Please review You can merge this pull request into a Git repository by running: $ git pull https://gith

[jira] [Commented] (APEXMALHAR-2010) Add transform operator

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201426#comment-15201426 ] ASF GitHub Bot commented on APEXMALHAR-2010: Github user sandeepdeshmukh

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201227#comment-15201227 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

Re: Changes to JdbcOutputOperator

2016-03-19 Thread Bhupesh Chawda
Hi, I have made the changes as discussed in previous emails. Here is the pull request: https://github.com/apache/incubator-apex-malhar/pull/215 Please help review the changes. Thanks ~Bhupesh On Tue, Mar 15, 2016 at 6:57 PM, Bhupesh Chawda wrote: > Hi, > > Here are some of the details on what

[jira] [Commented] (APEXMALHAR-2013) HDFS output module for file copy

2016-03-19 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201159#comment-15201159 ] ASF GitHub Bot commented on APEXMALHAR-2013: Github user DT-Priyanka comm

Stack overflow errors when launching job

2016-03-19 Thread Ganelin, Ilya
Hi guys – I’m running into a very frustrating issue where certain DAG configurations cause the following error log (attached). When this happens, my application even fails to launch. This does not seem to be a YARN issue since this occurs even with a relatively small number of partitions/memory.

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2011-2012 Avro to P...

2016-03-19 Thread devtagare
Github user devtagare commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/211#discussion_r56626092 --- Diff: contrib/src/main/java/com/datatorrent/contrib/avro/AvroRecordHelper.java --- @@ -114,6 +107,7 @@ private static Object convertAnd

Re: [MENTORS] Voting rules / bylaws draft

2016-03-19 Thread P. Taylor Goetz
Just to be clear, it’s not strictly necessary to establish bylaws in order to graduate. The “tasked with creating bylaws” section of graduation resolutions isn’t really necessary, but gets included due to copy/paste. Bylaws become necessary when you deviate from standard ASF practices. Most vot

[jira] [Commented] (APEXMALHAR-1953) Add generic (insert, update, delete) support to JDBC Output Operator

2016-03-19 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199483#comment-15199483 ] Bhupesh Chawda commented on APEXMALHAR-1953: Discussions also include the

  1   2   3   >