Re: APEXCORE-619 Recovery windowId in future during application relaunch.

2017-03-07 Thread Tushar Gosavi
Thank you all for the feedback. Some of the useful output operator can be stateless, they push data received in a window to output store. for example KafkaOutputOperator/JDBCOutputOperator, or the output stores where writes are idempotent, which covers most of the key-value stores. I was going to

Re: License information

2017-03-07 Thread Aniruddha Thombare
Hi, You wan't to include that code in your application or Malhar? License text for others: ImageJ is being developed at the National Institutes of Health by an employee of the Federal Government in the course of his official duties. Pursuant to Title 17, Section 105 of the *United States Code*,

Re: License information

2017-03-07 Thread Aditya gholba
Sorry for the ambiguity. The application I am making is with the intention of merging it with Malhar. Thanks, Aditya On 07-Mar-2017 2:34 PM, "Aniruddha Thombare" wrote: > Hi, > > You wan't to include that code in your application or Malhar? > > License text for others: > > ImageJ is being devel

Re: License information

2017-03-07 Thread Aniruddha Thombare
Hi, I think this answers it: https://www.apache.org/legal/resolved.html#can-works-placed-in-the-public-domain-be-included-in-apache-products Also check if the library includes more libraries and their licenses. https://www.apache.org/legal/resolved.html#concurrent Can someone else comment too?

[jira] [Commented] (APEXMALHAR-2364) Add user documentation for S3OutputModule

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899482#comment-15899482 ] ASF GitHub Bot commented on APEXMALHAR-2364: Github user asfgit closed th

[jira] [Resolved] (APEXMALHAR-2364) Add user documentation for S3OutputModule

2017-03-07 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munagala V. Ramanath resolved APEXMALHAR-2364. -- Resolution: Fixed Fix Version/s: 3.7.0 Merged > Add user do

[GitHub] apex-malhar pull request #563: Renamed demos to examples in apex-malhar.

2017-03-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/563 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] apex-malhar pull request #529: APEXMALHAR-2364 Added Documetation for S3Outp...

2017-03-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/529 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (APEXCORE-408) Ability to schedule Sub-DAG from running application

2017-03-07 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899550#comment-15899550 ] Thomas Weise commented on APEXCORE-408: --- Please remove the word documents and let'

[jira] [Updated] (APEXMALHAR-2233) Consolidate example apps into malhar apps

2017-03-07 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2233: - Fix Version/s: 3.7.0 > Consolidate example apps into malhar apps > --

[jira] [Updated] (APEXCORE-408) Ability to schedule Sub-DAG from running application

2017-03-07 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi updated APEXCORE-408: --- Attachment: (was: dag.docx) > Ability to schedule Sub-DAG from running application > ---

[jira] [Updated] (APEXCORE-408) Ability to schedule Sub-DAG from running application

2017-03-07 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi updated APEXCORE-408: --- Attachment: (was: dag.docx) > Ability to schedule Sub-DAG from running application > ---

[jira] [Created] (APEXCORE-661) Raise StramEvent for blocked operator

2017-03-07 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXCORE-661: -- Summary: Raise StramEvent for blocked operator Key: APEXCORE-661 URL: https://issues.apache.org/jira/browse/APEXCORE-661 Project: Apache Apex Core Issue

[jira] [Commented] (APEXCORE-408) Ability to schedule Sub-DAG from running application

2017-03-07 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899916#comment-15899916 ] Tushar Gosavi commented on APEXCORE-408: removed word document, and added link t

[jira] [Commented] (APEXMALHAR-2434) JMSTransactionableStore uses Session.createQueue() which fails

2017-03-07 Thread Sanjay M Pujare (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899917#comment-15899917 ] Sanjay M Pujare commented on APEXMALHAR-2434: - [~oliverwnk] Pls give your

[jira] [Created] (APEXCORE-662) Raise StramEvent for heartbeat miss

2017-03-07 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXCORE-662: -- Summary: Raise StramEvent for heartbeat miss Key: APEXCORE-662 URL: https://issues.apache.org/jira/browse/APEXCORE-662 Project: Apache Apex Core Issue Ty

[jira] [Updated] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2422: Affects Version/s: 3.5.0 3.6.0 > WindowDataManager not recovering as exp

[jira] [Updated] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2422: Affects Version/s: 3.7.0 > WindowDataManager not recovering as expected on HDFS > -

[jira] [Created] (APEXMALHAR-2435) specify KeyedWindowedOperatorBenchmarkApp application name

2017-03-07 Thread bright chen (JIRA)
bright chen created APEXMALHAR-2435: --- Summary: specify KeyedWindowedOperatorBenchmarkApp application name Key: APEXMALHAR-2435 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2435 Project: Apac

[jira] [Commented] (APEXMALHAR-2435) specify KeyedWindowedOperatorBenchmarkApp application name

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900471#comment-15900471 ] ASF GitHub Bot commented on APEXMALHAR-2435: GitHub user brightchen opene

[GitHub] apex-malhar pull request #574: APEXMALHAR-2435 #resolve #comment specify Key...

2017-03-07 Thread brightchen
GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/574 APEXMALHAR-2435 #resolve #comment specify KeyedWindowedOperatorBenchm… …arkApp application name You can merge this pull request into a Git repository by running: $ git pull https://

[GitHub] apex-malhar pull request #560: APEXMALHAR-2422 During recovery don't relay o...

2017-03-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/560 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900616#comment-15900616 ] ASF GitHub Bot commented on APEXMALHAR-2422: Github user asfgit closed th

[jira] [Updated] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2422: - Affects Version/s: (was: 3.7.0) > WindowDataManager not recovering as expected on

[jira] [Resolved] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved APEXMALHAR-2422. -- Resolution: Fixed Fix Version/s: 3.7.0 > WindowDataManager not recovering as

[jira] [Commented] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-03-07 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900620#comment-15900620 ] Thomas Weise commented on APEXMALHAR-2422: -- Please create a JIRA for test wi

[GitHub] apex-core pull request #436: APEXCORE-456 - Explicitly limit Server.Subscrib...

2017-03-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/436 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Commented] (APEXCORE-456) Explicitly limit Server.Subscriber to one way communication

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900632#comment-15900632 ] ASF GitHub Bot commented on APEXCORE-456: - Github user asfgit closed the pull re

[jira] [Resolved] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-03-07 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh resolved APEXMALHAR-2419. - Resolution: Done > KafkaSinglePortExactlyOnceOutputOperator fails on recovery > -

[GitHub] apex-malhar pull request #575: Remove unused variable and warning.

2017-03-07 Thread tweise
GitHub user tweise opened a pull request: https://github.com/apache/apex-malhar/pull/575 Remove unused variable and warning. @sandeshh review You can merge this pull request into a Git repository by running: $ git pull https://github.com/tweise/apex-malhar fswal-touchup Altern

[jira] [Created] (APEXMALHAR-2436) Make fileSystemWal unit tests to use MiniDFSCluster

2017-03-07 Thread Sandesh (JIRA)
Sandesh created APEXMALHAR-2436: --- Summary: Make fileSystemWal unit tests to use MiniDFSCluster Key: APEXMALHAR-2436 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2436 Project: Apache Apex Malhar

Re: License information

2017-03-07 Thread Aditya gholba
Thanks Aniruddha! -Aditya. On Tue, Mar 7, 2017 at 2:51 PM, Aniruddha Thombare < anirud...@datatorrent.com> wrote: > Hi, > > I think this answers it: > https://www.apache.org/legal/resolved.html#can-works- > placed-in-the-public-domain-be-included-in-apache-products > > Also check if the library

[GitHub] apex-malhar pull request #575: Remove unused variable and warning.

2017-03-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/575 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (APEXCORE-663) Application restart not working.

2017-03-07 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXCORE-663: -- Summary: Application restart not working. Key: APEXCORE-663 URL: https://issues.apache.org/jira/browse/APEXCORE-663 Project: Apache Apex Core Issue Type:

[jira] [Commented] (APEXCORE-663) Application restart not working.

2017-03-07 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900760#comment-15900760 ] Tushar Gosavi commented on APEXCORE-663: I believe this was introduced through 3

[jira] [Commented] (APEXCORE-592) Returning description field in defaultProperties during apex cli call get-app-package-info

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900763#comment-15900763 ] ASF GitHub Bot commented on APEXCORE-592: - Github user ajaygit158 closed the pul

[GitHub] apex-core pull request #464: APEXCORE-592 Return additional parameter descri...

2017-03-07 Thread ajaygit158
GitHub user ajaygit158 reopened a pull request: https://github.com/apache/apex-core/pull/464 APEXCORE-592 Return additional parameter description from properties.xml in defaultProperties for get-app-package-info command Sample data and output : **properties.xml** dt.ap

[GitHub] apex-core pull request #464: APEXCORE-592 Return additional parameter descri...

2017-03-07 Thread ajaygit158
Github user ajaygit158 closed the pull request at: https://github.com/apache/apex-core/pull/464 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXCORE-592) Returning description field in defaultProperties during apex cli call get-app-package-info

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900764#comment-15900764 ] ASF GitHub Bot commented on APEXCORE-592: - GitHub user ajaygit158 reopened a pul

[jira] [Assigned] (APEXCORE-663) Application restart not working.

2017-03-07 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh reassigned APEXCORE-663: Assignee: Sandesh > Application restart not working. > > >

Malhar build failing

2017-03-07 Thread Chinmay Kolhatkar
Hi, I'm finding malhar build to fail because of rat check failing in demos/ folder. Moreover, I cannot see demos/ folder in https://github.com/apache/apex-malhar I want to confirm whether its in apache repository itself OR its my environment issue. Please reply. Thanks, Chinmay.

[jira] [Created] (APEXMALHAR-2437) Malhar build - ratcheck failing

2017-03-07 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2437: - Summary: Malhar build - ratcheck failing Key: APEXMALHAR-2437 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2437 Project: Apache Apex Malhar

Re: Malhar build failing

2017-03-07 Thread Thomas Weise
demos -> examples You probably need to remove the leftover local demo folder with its target directories. Thomas On Tue, Mar 7, 2017 at 10:10 PM, Chinmay Kolhatkar wrote: > Hi, > > I'm finding malhar build to fail because of rat check failing in demos/ > folder. > > Moreover, I cannot see dem

Re: Malhar build failing

2017-03-07 Thread Chinmay Kolhatkar
Got it... After removing demos folder, its seems to run fine. On Wed, Mar 8, 2017 at 11:58 AM, Thomas Weise wrote: > demos -> examples > > You probably need to remove the leftover local demo folder with its target > directories. > > Thomas > > > On Tue, Mar 7, 2017 at 10:10 PM, Chinmay Kolhatkar

[jira] [Closed] (APEXMALHAR-2437) Malhar build - ratcheck failing

2017-03-07 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar closed APEXMALHAR-2437. - Resolution: Invalid > Malhar build - ratcheck failing > -

Watermark tuples in Apex

2017-03-07 Thread Bhupesh Chawda
Hi All, Watermark tuples in Apex are very tightly coupled to event time processing. For this reason, usually they are modeled as having a timestamp. public interface WatermarkTuple { long getTimestamp(); } Even though, watermarks are meant for such time related processing, I think we should ex

[GitHub] apex-site pull request #68: Updating malhar demos -> examples

2017-03-07 Thread chinmaykolhatkar
GitHub user chinmaykolhatkar opened a pull request: https://github.com/apache/apex-site/pull/68 Updating malhar demos -> examples @tweise Please merge. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chinmaykolhatkar/apex-site pat

[GitHub] apex-core pull request #483: APEXCORE-663 During restart fallback to old way...

2017-03-07 Thread sandeshh
GitHub user sandeshh opened a pull request: https://github.com/apache/apex-core/pull/483 APEXCORE-663 During restart fallback to old way of constructing the l… …ist of containers as YARN return empty list of containers @vrozov @tweise please review You can merge this pu

[jira] [Commented] (APEXCORE-663) Application restart not working.

2017-03-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900858#comment-15900858 ] ASF GitHub Bot commented on APEXCORE-663: - GitHub user sandeshh opened a pull re