Re: Java stream api

2016-04-04 Thread Siyuan Hua
I have collected some open topics/questions for discussion already from folks who already reviewed the code 1.The name, name of the Stream and the StreamSource 2. Build dag in an incremental way vs lazy population. Incremental way is easier to implement (what I did right now) and it create one ed

[jira] [Commented] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225759#comment-15225759 ] ASF GitHub Bot commented on APEXMALHAR-2043: Github user asfgit closed th

[GitHub] incubator-apex-malhar pull request: MLHR-1914: Add base operators ...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/142 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Resolved] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXMALHAR-2043. -- Resolution: Fixed > Update checkstyle plugin declaration to use apex-codestyle-conf

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2043 using property...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/231 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-apex-malhar pull request: MLHR-1881 topic & zookeeper ba...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/156 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-apex-malhar pull request: Committing files for additiona...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/153 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Java stream api

2016-04-04 Thread Siyuan Hua
Hi community, I have submitted my first commit of stream api into my public repository here https://github.com/siyuanh/incubator-apex-malhar/tree/stream You can think this is the prototype of the Java Stream API proposal here https://docs.google.com/document/d/163LmQjX860b61NDe3ZzR0hRTPtE-4GF0iH

Re: Podling Report Reminder - April 2016

2016-04-04 Thread Thomas Weise
Report posted to Wiki: https://wiki.apache.org/incubator/April2016 Thanks On Sun, Apr 3, 2016 at 10:39 AM, Chris Nauroth wrote: > +1. Thank you, Thomas. I'll add my mentor sign-off after this makes it > into wiki. > > --Chris Nauroth > > > > > On 4/2/16, 10:45 PM, "Thomas Weise" wrote: > >

Re: Bandwidth control for Input operators in Apex

2016-04-04 Thread Priyanka Gugale
Okay so I will open the pull request soon. -Priyanka On Wed, Mar 23, 2016 at 1:35 PM, Yogi Devendra wrote: > This looks OK. Let us build it incrementally. > > ~ Yogi > > On 23 March 2016 at 13:24, Sandeep Deshmukh > wrote: > > > I would suggest that we go ahead with design as suggested by Priy

Re: Spooled Data Structures

2016-04-04 Thread Timothy Farkas
Hi Ilya, After the Map or List is created it will work like a normal in memory java List or Map. We can simplify instantiating the object further though by automatically generating the Key Prefixes in the StateFactory object and only requiring the bucket to be specified once on the StateFactory li

RE: Spooled Data Structures

2016-04-04 Thread Ganelin, Ilya
Could this be an opportunity to abstract away the bucket/key relation? This is a rather cumbersome mechanism. For the user, having a primary and secondary key in this instance seems like it makes it more difficult to use the system easily. Sent with Good (www.good.com)

[jira] [Commented] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225677#comment-15225677 ] ASF GitHub Bot commented on APEXMALHAR-2043: Github user chandnisingh com

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2043 using property...

2016-04-04 Thread chandnisingh
Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/231#discussion_r58486048 --- Diff: library/pom.xml --- @@ -35,6 +35,7 @@ Apache Apex Malhar (incubating) Library +false --- End di

[jira] [Comment Edited] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225655#comment-15225655 ] Thomas Weise edited comment on APEXMALHAR-2026 at 4/5/16 4:12 AM: -

[jira] [Commented] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225655#comment-15225655 ] Thomas Weise commented on APEXMALHAR-2026: -- How will the key object be seria

[jira] [Commented] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225639#comment-15225639 ] ASF GitHub Bot commented on APEXMALHAR-2043: Github user tweise commented

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2043 using property...

2016-04-04 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/231#discussion_r58483684 --- Diff: library/pom.xml --- @@ -35,6 +35,7 @@ Apache Apex Malhar (incubating) Library +false --- End diff --

[GitHub] incubator-apex-core pull request: APEXCORE-411 support restarting ...

2016-04-04 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/294#discussion_r58482984 --- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java --- @@ -210,14 +239,67 @@ public StramAppLauncher(FileSystem fs,

[jira] [Commented] (APEXCORE-411) Restart app without specifying app package

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225617#comment-15225617 ] ASF GitHub Bot commented on APEXCORE-411: - Github user tweise commented on a dif

[GitHub] incubator-apex-core pull request: APEXCORE-411 support restarting ...

2016-04-04 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/294#discussion_r58482768 --- Diff: engine/src/main/java/com/datatorrent/stram/cli/DTCli.java --- @@ -1861,49 +1862,60 @@ public void execute(String[] args, ConsoleReader r

[jira] [Commented] (APEXCORE-411) Restart app without specifying app package

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225611#comment-15225611 ] ASF GitHub Bot commented on APEXCORE-411: - Github user tweise commented on a dif

[jira] [Resolved] (APEXCORE-409) Document json application format

2016-04-04 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan resolved APEXCORE-409. Resolution: Fixed Fix Version/s: 3.4.0 > Document json application format > ---

Re: Removing devel-3 branch from malhar repo

2016-04-04 Thread Thomas Weise
The devel-3 branch has been removed. Finally! On Sun, Mar 27, 2016 at 10:27 PM, Sandeep Deshmukh wrote: > +1. We need to move out of devel-3 branch asap. > > Regards, > Sandeep > > On Fri, Mar 25, 2016 at 10:40 PM, David Yan wrote: > > > +1. Let's get rid of it! > > On Mar 25, 2016 9:36 AM, "Th

[jira] [Resolved] (APEXMALHAR-1969) Add idempotency support to 0.9 KafkaInputOperator

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXMALHAR-1969. -- Resolution: Fixed > Add idempotency support to 0.9 KafkaInputOperator > ---

[jira] [Updated] (APEXMALHAR-1969) Add idempotency support to 0.9 KafkaInputOperator

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-1969: - Fix Version/s: 3.4.0 > Add idempotency support to 0.9 KafkaInputOperator > --

[jira] [Commented] (APEXMALHAR-1969) Add idempotency support to 0.9 KafkaInputOperator

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225463#comment-15225463 ] Thomas Weise commented on APEXMALHAR-1969: -- https://github.com/apache/incuba

[GitHub] incubator-apex-malhar pull request: Add Idempotent support the new...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225411#comment-15225411 ] ASF GitHub Bot commented on APEXMALHAR-2043: GitHub user chandnisingh ope

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2043 using property...

2016-04-04 Thread chandnisingh
GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/231 APEXMALHAR-2043 using property logViolationsToConsole instead of cons… …oleOutput You can merge this pull request into a Git repository by running: $ git pull https://gi

Spooled Data Structures

2016-04-04 Thread Timothy Farkas
Hi All, I'll be working on implementing Spooled Data structures. The overall proposal is the following: Overall design proposal: Spooled Data structures will be designed to work ontop of a key value store. Spooled data structures would be used as a component inside of an operator. The way the co

[jira] [Comment Edited] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Timothy Farkas (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225391#comment-15225391 ] Timothy Farkas edited comment on APEXMALHAR-2026 at 4/5/16 12:36 AM: --

[jira] [Commented] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Timothy Farkas (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225391#comment-15225391 ] Timothy Farkas commented on APEXMALHAR-2026: Overall design proposal: Sp

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225383#comment-15225383 ] ASF GitHub Bot commented on APEXCORE-415: - Github user davidyan74 commented on t

[jira] [Reopened] (APEXMALHAR-2043) Update checkstyle plugin declaration to use apex-codestyle-config artifact

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh reopened APEXMALHAR-2043: --- In Malhar we were enabling/disabling logging to console with the property `consoleOu

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread davidyan74
Github user davidyan74 commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205560349 @ilooner OK --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[jira] [Updated] (APEXCORE-420) When operator is declared @Stateless, the operator does not start correctly when restarting from a terminated app

2016-04-04 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan updated APEXCORE-420: --- Description: The operator will get stuck initially, and STRAM marks it blocked because the window ID

[jira] [Updated] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXCORE-419: -- Fix Version/s: 3.4.0 > When operator is blocked, print out a warning instead of debug > --

[jira] [Resolved] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXCORE-419. --- Resolution: Fixed > When operator is blocked, print out a warning instead of debug > ---

[jira] [Created] (APEXCORE-420) When operator is declared @Stateless, the operator does not start correctly when restarting from a terminated app

2016-04-04 Thread David Yan (JIRA)
David Yan created APEXCORE-420: -- Summary: When operator is declared @Stateless, the operator does not start correctly when restarting from a terminated app Key: APEXCORE-420 URL: https://issues.apache.org/jira/browse

[jira] [Commented] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225377#comment-15225377 ] ASF GitHub Bot commented on APEXCORE-419: - Github user asfgit closed the pull re

[GitHub] incubator-apex-core pull request: APEXCORE-419 #resolve when opera...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-core/pull/293 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225376#comment-15225376 ] ASF GitHub Bot commented on APEXCORE-415: - Github user ilooner commented on the

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread ilooner
Github user ilooner commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205558086 @PramodSSImmaneni @davidyan74 I have added tests. @davidyan74 I don't think your suggestion would work because it does not account for the ProcessMode and

[jira] [Commented] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Timothy Farkas (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225350#comment-15225350 ] Timothy Farkas commented on APEXMALHAR-2026: The use case of a map of lis

[jira] [Assigned] (APEXCORE-411) Restart app without specifying app package

2016-04-04 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan reassigned APEXCORE-411: -- Assignee: David Yan > Restart app without specifying app package > --

[jira] [Commented] (APEXCORE-411) Restart app without specifying app package

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225270#comment-15225270 ] ASF GitHub Bot commented on APEXCORE-411: - GitHub user davidyan74 opened a pull

[GitHub] incubator-apex-core pull request: APEXCORE-411 support restarting ...

2016-04-04 Thread davidyan74
GitHub user davidyan74 opened a pull request: https://github.com/apache/incubator-apex-core/pull/294 APEXCORE-411 support restarting app without app package or jar files @tweise please review You can merge this pull request into a Git repository by running: $ git pull https://g

[jira] [Commented] (APEXCORE-339) Support ability to tag operators as idempotent or non-idempotent

2016-04-04 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225223#comment-15225223 ] Pramod Immaneni commented on APEXCORE-339: -- Follow development here https://git

[jira] [Commented] (APEXMALHAR-2026) Spooled Datastructures

2016-04-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225192#comment-15225192 ] Thomas Weise commented on APEXMALHAR-2026: -- I believe #2 can be is supported

CEP model

2016-04-04 Thread Rodrigo Abreu
Hi, Siyuan mentioned that we definitely want to have some CEP model supported. Translate a CEP model explained here ( https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf) or Oracle CEP to Apex DAG would make many developers' life easier to use Apex. I also believe this can make the

Re: Contributing to Apache Apex

2016-04-04 Thread Rodrigo Abreu
Hi Siyuan, Thank you. I'm just back from my PTO. It makes complete sense to have a CEP model supported. I'm starting an email thread on it. Thanks, Rodrigo Abreu. On Thu, Mar 24, 2016 at 1:26 PM Siyuan Hua wrote: > Rodrigo, > > You are very welcome to the community. > We definitely want to h

[jira] [Updated] (APEXMALHAR-2025) Move FileLineInputOperator out of AbstractFileInputOperator

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXMALHAR-2025: -- Fix Version/s: 3.4.0 > Move FileLineInputOperator out of AbstractFileInputOperator

[jira] [Resolved] (APEXMALHAR-2025) Move FileLineInputOperator out of AbstractFileInputOperator

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh resolved APEXMALHAR-2025. --- Resolution: Fixed > Move FileLineInputOperator out of AbstractFileInputOperator >

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2025 Move FileLineI...

2016-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXMALHAR-2025) Move FileLineInputOperator out of AbstractFileInputOperator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225009#comment-15225009 ] ASF GitHub Bot commented on APEXMALHAR-2025: Github user asfgit closed th

[jira] [Commented] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224944#comment-15224944 ] ASF GitHub Bot commented on APEXCORE-419: - Github user tweise commented on the p

[GitHub] incubator-apex-core pull request: APEXCORE-419 #resolve when opera...

2016-04-04 Thread tweise
Github user tweise commented on the pull request: https://github.com/apache/incubator-apex-core/pull/293#issuecomment-205469590 Please check what happens after this message is logged. If the affected operators are reset, then why do we need this message? --- If your project is set u

[GitHub] incubator-apex-core pull request: APEXCORE-419 #resolve when opera...

2016-04-04 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/incubator-apex-core/pull/293#discussion_r58437448 --- Diff: engine/src/main/java/com/datatorrent/stram/StreamingContainerManager.java --- @@ -1911,12 +1911,17 @@ public void updateRecoveryCheckpoi

[jira] [Commented] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224941#comment-15224941 ] ASF GitHub Bot commented on APEXCORE-419: - Github user tweise commented on a dif

[jira] [Commented] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224903#comment-15224903 ] ASF GitHub Bot commented on APEXCORE-419: - GitHub user davidyan74 opened a pull

[GitHub] incubator-apex-core pull request: APEXCORE-419 #resolve when opera...

2016-04-04 Thread davidyan74
GitHub user davidyan74 opened a pull request: https://github.com/apache/incubator-apex-core/pull/293 APEXCORE-419 #resolve when operator is marked blocked, print out a warning instead of debug @PramodSSImmaneni please merge You can merge this pull request into a Git repository by r

[jira] [Created] (APEXCORE-419) When operator is blocked, print out a warning instead of debug

2016-04-04 Thread David Yan (JIRA)
David Yan created APEXCORE-419: -- Summary: When operator is blocked, print out a warning instead of debug Key: APEXCORE-419 URL: https://issues.apache.org/jira/browse/APEXCORE-419 Project: Apache Apex Cor

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224737#comment-15224737 ] ASF GitHub Bot commented on APEXCORE-415: - Github user sandeshh commented on the

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread sandeshh
Github user sandeshh commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205434534 @PramodSSImmaneni I am for having both the changes in. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205432955 @sandeshh as @ilooner mentioned this shouldn't be happening in the first place so I don't think we would need a flag --- If your project is set u

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224731#comment-15224731 ] ASF GitHub Bot commented on APEXCORE-415: - Github user PramodSSImmaneni commente

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224726#comment-15224726 ] ASF GitHub Bot commented on APEXCORE-415: - Github user ilooner commented on the

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread ilooner
Github user ilooner commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205431850 @PramodSSImmaneni will add test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224713#comment-15224713 ] ASF GitHub Bot commented on APEXCORE-415: - Github user PramodSSImmaneni commente

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224710#comment-15224710 ] ASF GitHub Bot commented on APEXCORE-415: - Github user PramodSSImmaneni commente

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205429004 @davidyan74 Will need a lot more refactoring to handle in Node as control tuples are being processed in the specific Node types. --- If your proj

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224705#comment-15224705 ] ASF GitHub Bot commented on APEXCORE-415: - Github user sandeshh commented on the

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205428578 @ilooner Change looks good, can you add a test. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread sandeshh
Github user sandeshh commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205428183 How about having the flag in CheckPoint function which decides whether to overwrite the existing checkpoints or not ? --- If your project is set up for i

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224694#comment-15224694 ] ASF GitHub Bot commented on APEXCORE-415: - Github user davidyan74 commented on t

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread davidyan74
Github user davidyan74 commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205426991 @ilooner @PramodSSImmaneni I see. Then shouldn't this change be in Node? --- If your project is set up for it, you can reply to this email and have you

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224689#comment-15224689 ] ASF GitHub Bot commented on APEXCORE-415: - Github user ilooner commented on the

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224687#comment-15224687 ] ASF GitHub Bot commented on APEXCORE-415: - Github user PramodSSImmaneni commente

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread ilooner
Github user ilooner commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205426336 @davidyan74 There was a similar bug in Generic Node which I fixed and got merged before. The problem happens when The Operator CheckpointWindowCount is a m

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205426102 @davidyan74 I think this check happens for other nodes but is not happening for InputNode --- If your project is set up for it, you can reply to

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224680#comment-15224680 ] ASF GitHub Bot commented on APEXCORE-415: - Github user davidyan74 commented on t

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread davidyan74
Github user davidyan74 commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205424902 @ilooner Does this only happen in the InputNode? How about overriding the checkpoint() method in the InputNode for setting and checking for lastCheckpoi

[jira] [Commented] (APEXMALHAR-2044) Exception in AbstractFileOutputOperator during write.

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224668#comment-15224668 ] Chandni Singh commented on APEXMALHAR-2044: --- The exception is being thrown

[jira] [Commented] (APEXMALHAR-2044) Exception in AbstractFileOutputOperator during write.

2016-04-04 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224650#comment-15224650 ] Tushar Gosavi commented on APEXMALHAR-2044: --- Malhar version is 3.3.0-incuba

[jira] [Commented] (APEXMALHAR-2044) Exception in AbstractFileOutputOperator during write.

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224612#comment-15224612 ] Chandni Singh commented on APEXMALHAR-2044: --- which version of Malhar? > Ex

[jira] [Commented] (APEXCORE-415) Input Operator Can Double Checkpoint if Operator CheckpointWindowCount is greater than DAG CheckpointWindowCount

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224608#comment-15224608 ] ASF GitHub Bot commented on APEXCORE-415: - Github user ilooner commented on the

[GitHub] incubator-apex-core pull request: APEXCORE-415 #resolve #comment F...

2016-04-04 Thread ilooner
Github user ilooner commented on the pull request: https://github.com/apache/incubator-apex-core/pull/292#issuecomment-205413237 @davidyan74 Please review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[jira] [Assigned] (APEXMALHAR-2044) Exception in AbstractFileOutputOperator during write.

2016-04-04 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh reassigned APEXMALHAR-2044: - Assignee: Chandni Singh > Exception in AbstractFileOutputOperator during wri

[jira] [Created] (APEXMALHAR-2044) Exception in AbstractFileOutputOperator during write.

2016-04-04 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXMALHAR-2044: - Summary: Exception in AbstractFileOutputOperator during write. Key: APEXMALHAR-2044 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2044 Project: Apach

[jira] [Commented] (APEXCORE-316) Upstream operator continues to use DefaultStatefulStreamCode even when different StreamCodec set

2016-04-04 Thread pradeep kumbhar (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223896#comment-15223896 ] pradeep kumbhar commented on APEXCORE-316: -- Hi [~timothyfarkas], I have not obs

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223881#comment-15223881 ] ASF GitHub Bot commented on APEXMALHAR-2014: GitHub user shubham-pathak22

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-04-04 Thread shubham-pathak22
GitHub user shubham-pathak22 reopened a pull request: https://github.com/apache/incubator-apex-malhar/pull/230 APEXMALHAR-2014 added parquet reader You can merge this pull request into a Git repository by running: $ git pull https://github.com/shubham-pathak22/incubator-apex-m

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-04-04 Thread shubham-pathak22
Github user shubham-pathak22 closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/230 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223873#comment-15223873 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user shubham-pathak22

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223865#comment-15223865 ] ASF GitHub Bot commented on APEXMALHAR-2014: GitHub user shubham-pathak22

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-04-04 Thread shubham-pathak22
GitHub user shubham-pathak22 opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/230 APEXMALHAR-2014 added parquet reader You can merge this pull request into a Git repository by running: $ git pull https://github.com/shubham-pathak22/incubator-apex-mal

[jira] [Commented] (APEXMALHAR-2014) ParquetReader operator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223863#comment-15223863 ] ASF GitHub Bot commented on APEXMALHAR-2014: Github user shubham-pathak22

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2014 added parquet ...

2016-04-04 Thread shubham-pathak22
Github user shubham-pathak22 closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[jira] [Commented] (APEXMALHAR-1980) Add metrics to Cassandra Input operator

2016-04-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223823#comment-15223823 ] ASF GitHub Bot commented on APEXMALHAR-1980: Github user DT-Priyanka clos

  1   2   >