Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Sandeep Deshmukh
+1 on making it apex feature. On 29-Mar-2016 12:22 pm, "Tushar Gosavi" wrote: > +1 for the idea, love to have this feature available for operators writing > to external systems. > > - Tushar. > > > On Tue, Mar 29, 2016 at 11:51 AM, Chinmay Kolhatkar > wrote: > > > Hi Ashwin, > > > > This is inde

Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Tushar Gosavi
+1 for the idea, love to have this feature available for operators writing to external systems. - Tushar. On Tue, Mar 29, 2016 at 11:51 AM, Chinmay Kolhatkar wrote: > Hi Ashwin, > > This is indeed a very required functionality. > > I think this is applicable for all three types of operators: I

Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Chinmay Kolhatkar
Hi Ashwin, This is indeed a very required functionality. I think this is applicable for all three types of operators: Input (e.g. HDFSInput, JDBCInput, etc), Process (e.g. Enrichment), Output (JDBCOutput, HDFSOutput, etc) I tried to do something similar: https://github.com/chinmaykolhatkar/incub

Re: Apache Apex (incubating) 3.2.1 patch release

2016-03-28 Thread Amol Kekre
+1. There are quite a few NPE and other bugs that should go out. Any others that can make it ? Thks Amol On Mon, Mar 28, 2016 at 9:40 PM, Thomas Weise wrote: > +1 for patch release in April. Several critical bug fixes that should be in > official release. > > On Mon, Mar 28, 2016 at 1:47 PM,

[jira] [Commented] (APEXMALHAR-2033) Streaming JSON parser

2016-03-28 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215476#comment-15215476 ] Chinmay Kolhatkar commented on APEXMALHAR-2033: --- Hi Devendra, I see Ja

Re: GSOC: AVSR in Apache Apex

2016-03-28 Thread Priyanka Gugale
Good. So you or anyone from Apex community would like to mentor this project? I would take care of rest of the things. -Priyanka On Mar 23, 2016 5:29 PM, "Atri Sharma" wrote: 1) You need to write to IPMC, not Google, for being a mentor. ASF will take your name as a mentor for ASF. 2) Yes, we can

[GitHub] incubator-apex-site pull request: Reorganize docs

2016-03-28 Thread tweise
Github user tweise commented on the pull request: https://github.com/apache/incubator-apex-site/pull/28#issuecomment-202712068 @amberarrow There are a few unexpected line breaks in the final page. Please check. --- If your project is set up for it, you can reply to this email and hav

[GitHub] incubator-apex-site pull request: Reorganize docs

2016-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-site/pull/28 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[jira] [Updated] (APEXMALHAR-2036) FS operator tests leave stray test files under target

2016-03-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2036: - Priority: Minor (was: Major) > FS operator tests leave stray test files under target

Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Mohit Jotwani
Dear Ashwin, The approach sounds good. I am assuming that this will be done for all the output data stores and not limited to JDBC. +1 Regards, Mohit On Tue, Mar 29, 2016 at 5:17 AM, Ashwin Chandra Putta < ashwinchand...@gmail.com> wrote: > There are many use cases in which we are writing tupl

[jira] [Commented] (APEXMALHAR-2036) FS operator tests leave stray test files under target

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215418#comment-15215418 ] ASF GitHub Bot commented on APEXMALHAR-2036: GitHub user chandnisingh ope

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2036 cleanup of int...

2016-03-28 Thread chandnisingh
GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/223 APEXMALHAR-2036 cleanup of intermediate test files You can merge this pull request into a Git repository by running: $ git pull https://github.com/chandnisingh/incubator-ap

[jira] [Updated] (APEXMALHAR-2036) FS operator tests leave stray test files under target

2016-03-28 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXMALHAR-2036: -- Description: Some of the tests of the following classes do not delete the files tha

Re: Apache Apex (incubating) 3.2.1 patch release

2016-03-28 Thread Thomas Weise
+1 for patch release in April. Several critical bug fixes that should be in official release. On Mon, Mar 28, 2016 at 1:47 PM, Vlad Rozov wrote: > There are 17 issues planned for 3.2.1 patch release > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20APEXCORE%20AND%20(fixVersion%20%

[jira] [Commented] (APEXMALHAR-2036) FS operator tests leave stray test files under target

2016-03-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215408#comment-15215408 ] Thomas Weise commented on APEXMALHAR-2036: -- What's the issue (description mi

[jira] [Updated] (APEXMALHAR-2036) FS operator tests leave stray test files under target

2016-03-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2036: - Summary: FS operator tests leave stray test files under target (was: FS operator tes

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215214#comment-15215214 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user devtagare commen

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215212#comment-15215212 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user devtagare commen

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-03-28 Thread devtagare
Github user devtagare commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/219#discussion_r57661266 --- Diff: contrib/src/main/java/com/datatorrent/contrib/parquet/ParquetFilePOJOReader.java --- @@ -0,0 +1,305 @@ +/** + * Licensed

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-03-28 Thread devtagare
Github user devtagare commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/219#discussion_r57661315 --- Diff: contrib/src/main/java/com/datatorrent/contrib/parquet/AbstractParquetFileReader.java --- @@ -0,0 +1,113 @@ +/** + * Licen

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-03-28 Thread devtagare
Github user devtagare commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/219#discussion_r57661028 --- Diff: contrib/src/main/java/com/datatorrent/contrib/parquet/ParquetFilePOJOReader.java --- @@ -0,0 +1,305 @@ +/** + * Licensed

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215206#comment-15215206 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user devtagare commen

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215204#comment-15215204 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user devtagare commen

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-03-28 Thread devtagare
Github user devtagare commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/219#discussion_r57660735 --- Diff: contrib/src/main/java/com/datatorrent/contrib/parquet/ParquetFilePOJOReader.java --- @@ -0,0 +1,305 @@ +/** + * Licensed

Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Ashwin Chandra Putta
Can you give more details Sandesh what you mean by bandwidth controller? Regards, Ashwin. On Mon, Mar 28, 2016 at 4:58 PM, Sandesh Hegde wrote: > Essentially we need bandwidth controller for output operators? not specific > to JDBC? > > On Mon, Mar 28, 2016 at 4:47 PM Ashwin Chandra Putta < > a

Re: proposal for reconciled JDBC output operator

2016-03-28 Thread Sandesh Hegde
Essentially we need bandwidth controller for output operators? not specific to JDBC? On Mon, Mar 28, 2016 at 4:47 PM Ashwin Chandra Putta < ashwinchand...@gmail.com> wrote: > There are many use cases in which we are writing tuples to external system > using JDBC etc. There are instances when the

[jira] [Updated] (APEXMALHAR-2037) Reconciled jdbc output operator with ability to spool to disk

2016-03-28 Thread Ashwin Chandra Putta (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Chandra Putta updated APEXMALHAR-2037: - Description: There are many use cases in which we are writing tuples

proposal for reconciled JDBC output operator

2016-03-28 Thread Ashwin Chandra Putta
There are many use cases in which we are writing tuples to external system using JDBC etc. There are instances when the external system might be slow and down for some time. In those cases, the current implementation of jdbc output operators fail and restart until the external system is up again. M

[jira] [Created] (APEXMALHAR-2037) Reconciled jdbc output operator with ability to spool to disk

2016-03-28 Thread Ashwin Chandra Putta (JIRA)
Ashwin Chandra Putta created APEXMALHAR-2037: Summary: Reconciled jdbc output operator with ability to spool to disk Key: APEXMALHAR-2037 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2037

[jira] [Commented] (APEXMALHAR-2031) Allow Window Data Manager to store data in a user specified directory

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215096#comment-15215096 ] ASF GitHub Bot commented on APEXMALHAR-2031: Github user asfgit closed th

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2031 enable WindowD...

2016-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/222 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXCORE-409) Document json application format

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215066#comment-15215066 ] ASF GitHub Bot commented on APEXCORE-409: - GitHub user davidyan74 opened a pull

[GitHub] incubator-apex-core pull request: APEXCORE-409 #resolve added docu...

2016-03-28 Thread davidyan74
GitHub user davidyan74 opened a pull request: https://github.com/apache/incubator-apex-core/pull/288 APEXCORE-409 #resolve added documentation for app specification using json @PramodSSImmaneni please review and merge You can merge this pull request into a Git repository by running

[jira] [Created] (APEXMALHAR-2036) FS operator tests leave stray test files under targer

2016-03-28 Thread Chandni Singh (JIRA)
Chandni Singh created APEXMALHAR-2036: - Summary: FS operator tests leave stray test files under targer Key: APEXMALHAR-2036 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2036 Project: Apach

[jira] [Commented] (APEXMALHAR-2031) Allow Window Data Manager to store data in a user specified directory

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215050#comment-15215050 ] ASF GitHub Bot commented on APEXMALHAR-2031: GitHub user chandnisingh ope

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2031 enable WindowD...

2016-03-28 Thread chandnisingh
GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/222 APEXMALHAR-2031 enable WindowDataManager to save window files under u… …ser configured path You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (APEXCORE-380) Idle time sleep time should increase from 0 to a configurable max value

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214917#comment-15214917 ] ASF GitHub Bot commented on APEXCORE-380: - GitHub user sandeshh opened a pull re

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread sandeshh
GitHub user sandeshh opened a pull request: https://github.com/apache/incubator-apex-core/pull/287 apexcore-407 adaptive spin millis for input operators I tested change with the benchmarking and I can clearly see the improvement in performance. It is similar to what was done before

[jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214893#comment-15214893 ] ASF GitHub Bot commented on APEXCORE-10: Github user ishark commented on a diff i

[GitHub] incubator-apex-core pull request: APEXCORE-10 #resolve Changes for...

2016-03-28 Thread ishark
Github user ishark commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/250#discussion_r57633606 --- Diff: engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlanConfiguration.java --- @@ -2435,7 +2435,7 @@ private void setAttribu

[jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214889#comment-15214889 ] ASF GitHub Bot commented on APEXCORE-10: Github user ishark commented on a diff i

[GitHub] incubator-apex-core pull request: APEXCORE-10 #resolve Changes for...

2016-03-28 Thread ishark
Github user ishark commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/250#discussion_r57633300 --- Diff: engine/src/main/java/com/datatorrent/stram/ResourceRequestHandler.java --- @@ -108,6 +113,23 @@ public void updateNodeReports(List node

[jira] [Commented] (APEXCORE-380) Idle time sleep time should increase from 0 to a configurable max value

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214885#comment-15214885 ] ASF GitHub Bot commented on APEXCORE-380: - Github user sandeshh closed the pull

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread sandeshh
Github user sandeshh closed the pull request at: https://github.com/apache/incubator-apex-core/pull/285 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Updated] (APEXCORE-130) Throwing A Runtime Exception In Setup Causes The Operator To Block

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-130: Affects Version/s: (was: 3.4.0) > Throwing A Runtime Exception In Setup Causes The Operator To

[jira] [Updated] (APEXCORE-350) STRAM's REST service sometimes returns duplicate and conflicting Content-Type headers

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-350: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-353) Buffer server may stop processing data

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-353: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-365) Buffer server handling for tuple length that exceeds data list block size

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-365: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-358) Make RPC timeout configurable

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-358: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-363) NPE in StreamingContainerManager

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-363: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-362) NPE in StreamingContainerManager

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-362: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-374) Block with positive reference count is found during buffer server purge

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-374: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-375) Container killed because of Out of Sequence tuple error.

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-375: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-385) Temp directories/files not always cleaned up when launching applications

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-385: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-391) AsyncFSStorageAgent creates tmp directory unnecessarily

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-391: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-398) Ack may not be delivered from buffer server to it's client

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-398: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

[jira] [Updated] (APEXCORE-410) Upgrade to netlet 1.2.1

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-410: Fix Version/s: 3.3.1 3.4.0 3.2.1 > Upgrade to netlet 1.2.1 >

[jira] [Updated] (APEXCORE-410) Upgrade to netlet 1.2.1

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-410: Affects Version/s: (was: 3.3.1) (was: 3.4.0)

Apache Apex (incubating) 3.2.1 patch release

2016-03-28 Thread Vlad Rozov
There are 17 issues planned for 3.2.1 patch release https://issues.apache.org/jira/issues/?jql=project%20%3D%20APEXCORE%20AND%20(fixVersion%20%3D%203.2.1%20or%20affectedVersion%20%3D%203.2.1) Should any other bug fixes be included into the release? Should we release in April?

[jira] [Updated] (APEXCORE-350) STRAM's REST service sometimes returns duplicate and conflicting Content-Type headers

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-350: Affects Version/s: 3.3.1 3.4.0 3.2.1

[jira] [Updated] (APEXCORE-385) Temp directories/files not always cleaned up when launching applications

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-385: Affects Version/s: 3.3.1 3.4.0 3.2.1

[jira] [Updated] (APEXCORE-391) AsyncFSStorageAgent creates tmp directory unnecessarily

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-391: Affects Version/s: 3.2.0 3.3.0 > AsyncFSStorageAgent creates tmp directory

[jira] [Updated] (APEXCORE-391) AsyncFSStorageAgent creates tmp directory unnecessarily

2016-03-28 Thread Vlad Rozov (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-391: Affects Version/s: 3.3.1 3.4.0 3.2.1 > AsyncFSStorag

[jira] [Created] (APEXMALHAR-2035) ManagedState use case- Add a deduper to Malhar which is backed by ManagedState

2016-03-28 Thread Chandni Singh (JIRA)
Chandni Singh created APEXMALHAR-2035: - Summary: ManagedState use case- Add a deduper to Malhar which is backed by ManagedState Key: APEXMALHAR-2035 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2035

[jira] [Created] (APEXCORE-410) Upgrade to netlet 1.2.1

2016-03-28 Thread Vlad Rozov (JIRA)
Vlad Rozov created APEXCORE-410: --- Summary: Upgrade to netlet 1.2.1 Key: APEXCORE-410 URL: https://issues.apache.org/jira/browse/APEXCORE-410 Project: Apache Apex Core Issue Type: Task Affec

[jira] [Commented] (APEXCORE-402) Provide SpscArrayQueue based implementation that uses Condition to notify publishing thread on not full condition

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214800#comment-15214800 ] ASF GitHub Bot commented on APEXCORE-402: - GitHub user vrozov opened a pull requ

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread sandeshh
Github user sandeshh commented on the pull request: https://github.com/apache/incubator-apex-core/pull/285#issuecomment-202564704 @vrozov I will squash the changes. Earlier the app was doing <71 tuples/second, after the change, it can do +10,00,000 tuples /secon

[GitHub] incubator-apex-core pull request: APEXCORE-402

2016-03-28 Thread vrozov
GitHub user vrozov opened a pull request: https://github.com/apache/incubator-apex-core/pull/286 APEXCORE-402 Provide SpscArrayQueue based implementation that uses Condition to notify publishing thread on not full condition. @tweise, @PramodSSImmaneni Please review. You ca

[jira] [Commented] (APEXMALHAR-1991) Move Dimensions Computation Classes to org.apache.apex.malhar package and Mark evolving

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214799#comment-15214799 ] ASF GitHub Bot commented on APEXMALHAR-1991: GitHub user brightchen opene

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1991 #resolve #comm...

2016-03-28 Thread brightchen
GitHub user brightchen opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/221 APEXMALHAR-1991 #resolve #comment Move Dimensions Computation Classes… … to org.apache.apex.malhar package You can merge this pull request into a Git repository by running:

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread vrozov
Github user vrozov commented on the pull request: https://github.com/apache/incubator-apex-core/pull/285#issuecomment-202561836 @sandeshh Please squash. What is the performance improvement? --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread davidyan74
Github user davidyan74 commented on the pull request: https://github.com/apache/incubator-apex-core/pull/285#issuecomment-202558591 @sandeshh it looks like it's a resource issue: https://s3.amazonaws.com/archive.travis-ci.org/jobs/119052755/log.txt Take a look at the exceptio

[jira] [Created] (APEXCORE-409) Document json application format

2016-03-28 Thread Pramod Immaneni (JIRA)
Pramod Immaneni created APEXCORE-409: Summary: Document json application format Key: APEXCORE-409 URL: https://issues.apache.org/jira/browse/APEXCORE-409 Project: Apache Apex Core Issue T

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/285#discussion_r57617120 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/InputNode.java --- @@ -97,8 +98,11 @@ public final void run() (

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/285#discussion_r57616145 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/InputNode.java --- @@ -97,8 +98,11 @@ public final void run() ((I

[jira] [Commented] (APEXCORE-380) Idle time sleep time should increase from 0 to a configurable max value

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214630#comment-15214630 ] ASF GitHub Bot commented on APEXCORE-380: - GitHub user sandeshh opened a pull re

[GitHub] incubator-apex-core pull request: apexcore-407 adaptive spin milli...

2016-03-28 Thread sandeshh
GitHub user sandeshh opened a pull request: https://github.com/apache/incubator-apex-core/pull/285 apexcore-407 adaptive spin millis for input operators I tested change with the benchmarking and I can clearly see the improvement in performance. It is similar to what was done before

[jira] [Assigned] (APEXMALHAR-1991) Move Dimensions Computation Classes to org.apache.apex.malhar package and Mark evolving

2016-03-28 Thread bright chen (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen reassigned APEXMALHAR-1991: --- Assignee: bright chen > Move Dimensions Computation Classes to org.apache.apex.m

[jira] [Commented] (APEXMALHAR-1996) Running a Storm topology on Apex

2016-03-28 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214620#comment-15214620 ] David Yan commented on APEXMALHAR-1996: --- The link leads to 404. Here's an upda

Re: Naming sugestion for HDFS output modules

2016-03-28 Thread Amol Kekre
Chinmay, Sure, do update the devel guidelines once we get an agreement on this thead. Thks Amol On Mon, Mar 28, 2016 at 11:18 AM, Chinmay Kolhatkar wrote: > @amol, that's a very explanation... Maybe it would be good point to add in > development best practices, if we have any. > > --- > Sent f

Re: Naming sugestion for HDFS output modules

2016-03-28 Thread Chinmay Kolhatkar
@amol, that's a very explanation... Maybe it would be good point to add in development best practices, if we have any. --- Sent from mobile. On 28 Mar 2016 10:52 p.m., "Amol Kekre" wrote: > I agree. Whether a resuable code component is module or operator is > internal to the implementations. The

[jira] [Created] (APEXMALHAR-2034) Avro File Input Operator

2016-03-28 Thread devendra tagare (JIRA)
devendra tagare created APEXMALHAR-2034: --- Summary: Avro File Input Operator Key: APEXMALHAR-2034 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034 Project: Apache Apex Malhar I

[jira] [Created] (APEXMALHAR-2033) Streaming JSON parser

2016-03-28 Thread devendra tagare (JIRA)
devendra tagare created APEXMALHAR-2033: --- Summary: Streaming JSON parser Key: APEXMALHAR-2033 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2033 Project: Apache Apex Malhar Issu

[jira] [Created] (APEXMALHAR-2032) MapReduce Input format support for File Splitter

2016-03-28 Thread devendra tagare (JIRA)
devendra tagare created APEXMALHAR-2032: --- Summary: MapReduce Input format support for File Splitter Key: APEXMALHAR-2032 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2032 Project: Apache

[RESULT] [VOTE] Graduate Apex from the Incubator

2016-03-28 Thread Pramod Immaneni
The voting is now closed. It has passed with 42 +1 votes and no 0 or -1 votes. The breakdown of the votes, in the order in which they were received under the two categories, is as follows. +1 (binding) Pramod Immaneni Thomas Weise David Yan Amol Kekre Ilya Ganglion Vlad Rozov Hitesh Shah Chris Na

[RESULT] [VOTE] Graduate Apex from the Incubator

2016-03-28 Thread Pramod Immaneni
The voting is now closed. It has passed with 42 +1 votes and no 0 or -1 votes. The breakdown of the votes, in the order in which they were received under the two categories, is as follows. +1 (binding) Pramod Immaneni Thomas Weise David Yan Amol Kekre Ilya Ganglion Vlad Rozov Hitesh Shah Chris Na

Re: Naming sugestion for HDFS output modules

2016-03-28 Thread Devendra Tagare
Is the plan to align the tuple writer with the org.apache.hadoop.mapred output formats ? https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/mapred/OutputFormat.html The advantage of this would be that Apex can be used for ETL's to write mapreduce compatible output files which can be used

Re: Naming sugestion for HDFS output modules

2016-03-28 Thread Amol Kekre
I agree. Whether a resuable code component is module or operator is internal to the implementations. The interface is "name+ports+properties". This is similar to a function signature in other languages. The name should convey the functional nature of the code and nothing more. Thks Amol On Mon, M

Re: [APEXCORE-107] Adding modules in DAG through property and json file.

2016-03-28 Thread Chinmay Kolhatkar
+1 for approach 2. --- Sent from mobile. On 23 Mar 2016 4:25 p.m., "Tushar Gosavi" wrote: > Hi All, > > I am planning provide support for adding modules into the json and property > file specification of DAG. We will go with the same syntax as discussed in > following mail thread. > > > https://

Re: Naming sugestion for HDFS output modules

2016-03-28 Thread Chinmay Kolhatkar
Sorry for delay in reply... Still catching up with emails.. I'm not sure whether we should have "Module" in the names. How about HDFSTupleWriter and HDFSFileWriter? Thanks, Chinmay. --- Sent from mobile. On 23 Mar 2016 4:49 p.m., "Yogi Devendra" wrote: > Hi, > > Currently, I am in the process

[jira] [Commented] (APEXMALHAR-1979) Ability to generate compiled class from given json schema

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214371#comment-15214371 ] ASF GitHub Bot commented on APEXMALHAR-1979: Github user chinmaykolhatkar

[GitHub] incubator-apex-site pull request: Reorganize docs

2016-03-28 Thread tweise
Github user tweise commented on the pull request: https://github.com/apache/incubator-apex-site/pull/28#issuecomment-202457900 This may be a good link to place on this page or elsewhere on the site for general ASF background: https://blogs.apache.org/foundation/entry/celebrati

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1979 Generate compi...

2016-03-28 Thread chinmaykolhatkar
Github user chinmaykolhatkar closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/181 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[jira] [Commented] (APEXMALHAR-1963) Add abstract operator for Async Processing

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214370#comment-15214370 ] ASF GitHub Bot commented on APEXMALHAR-1963: Github user chinmaykolhatkar

[jira] [Commented] (APEXMALHAR-1972) Create Expression Evaluator Support quasi-Java Expression Language

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214369#comment-15214369 ] ASF GitHub Bot commented on APEXMALHAR-1972: Github user chinmaykolhatkar

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1963 AbstractAsyncP...

2016-03-28 Thread chinmaykolhatkar
Github user chinmaykolhatkar closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/157 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-1972 Added support ...

2016-03-28 Thread chinmaykolhatkar
Github user chinmaykolhatkar closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[jira] [Closed] (APEXMALHAR-2015) Projection Operator

2016-03-28 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar closed APEXMALHAR-2015. - Resolution: Fixed Fix Version/s: 3.4.0 > Projection Operator > ---

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2015: Projection Op...

2016-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/217 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXMALHAR-2015) Projection Operator

2016-03-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214352#comment-15214352 ] ASF GitHub Bot commented on APEXMALHAR-2015: Github user asfgit closed th

  1   2   >