[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2082 Data Filter Op...

Github user chinmaykolhatkar commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/270#discussion_r64316099 --- Diff: library/src/main/java/com/datatorrent/lib/filter/FilterOperator.java --- @@ -0,0 +1,210 @@ +/** + * Licensed to the

[jira] [Commented] (APEXMALHAR-2082) Data Filter Operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297486#comment-15297486 ] ASF GitHub Bot commented on APEXMALHAR-2082: Github user chinmaykolhatkar

[GitHub] incubator-apex-core pull request: *Review only* Prototype of runni...

Github user sandeshh closed the pull request at: https://github.com/apache/incubator-apex-core/pull/340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (APEXMALHAR-2096) Add blockThreshold parameter to FSInputModule

[ https://issues.apache.org/jira/browse/APEXMALHAR-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297308#comment-15297308 ] ASF GitHub Bot commented on APEXMALHAR-2096: GitHub user DT-Priyanka open

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2096: Add property ...

GitHub user DT-Priyanka opened a pull request: https://github.com/apache/incubator-apex-malhar/pull/287 APEXMALHAR-2096: Add property blocksThreshold to limit input rate to FSInputModule You can merge this pull request into a Git repository by running: $ git pull https://gith

[jira] [Commented] (APEXMALHAR-2045) Add bandwidth control feature to Apex

[ https://issues.apache.org/jira/browse/APEXMALHAR-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297262#comment-15297262 ] ASF GitHub Bot commented on APEXMALHAR-2045: Github user DT-Priyanka comm

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2045: Adding bandwi...

Github user DT-Priyanka commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/279#discussion_r64301580 --- Diff: library/src/main/java/com/datatorrent/lib/bandwidth/BandwidthManager.java --- @@ -0,0 +1,137 @@ +/** + * Licensed to the

Re: Time controlled ingestion operator

Hi Sunil, The current implementation of JDBC input operator does not support a time-controlled behavior. As a reference on how to do polling reads there is a PR opened - https://github.com/apache/incubator-apex-malhar/pull/282 One of the solutions for the use-case using Apex features would be as

Re: Time controlled ingestion operator

Hi Sunil, I don't think we have exactly pause and resume feature in Malhar yet. But I can think of some other way to achieve this. We are adding bandwidth control feature to Malhar. The pull request is already opened for the same. This doe

Time controlled ingestion operator

All, We're writing an app for migration data from legacy JDBC system to Kafka. Legacy system as it is still being used in production, we would want to pause the JDBC poll input operator for configured hours. Does malhar ibrary has any operator that we can use for time controlled operator ? Or i

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1. - File Integrity: PASS - No Binary file in the package: PASS - Compilation: PASS - Unit tests: PASS On Tue, May 24, 2016 at 1:20 AM, David Yan wrote: > +1 (binding) > > - Build passed > - All unit tests passed > - Successfully ran pi demo with Apex Core 3.4.0. > > On Fri, May 20, 2016 at

Re: Proposal for Hive output module

I would suggest this, 1. Mark the *hiveStore* field in HiveOperator class as depricated. 2. Mark the getters and setters for hiveStore field as depricated, also in javadoc mention that you should use getStore/setStore instead. 3. Update the hiveStore getter setter code to actually set the store of

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1 (binding) - Build passed - All unit tests passed - Successfully ran pi demo with Apex Core 3.4.0. On Fri, May 20, 2016 at 8:49 AM, Thomas Weise wrote: > Dear Community, > > Please vote on the following Apache Apex Malhar 3.4.0 release candidate. > > This release follows core 3.4.0, resolves

[jira] [Commented] (APEXMALHAR-2094) Quantiles sketch operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296935#comment-15296935 ] Sandeep Narayanaswami commented on APEXMALHAR-2094: --- Sounds good. I

Re: Getting various statitstics from StreamingContainers

After connecting to the app user will run the following command. Users will select the container id, jdk tool and the arguments to the tool. Apex CLI api run-jdkTools "Container-id" "Tool-name" "Arguments" Output of the command is interpreted by the user. On Mon, May 23, 2016 at 11:31 AM Th

Re: Apache Beam Integration

Siouan - this is great guidance. I’m going to start with a deep review of the Beam API and the existing Apex API to identify overlap and will iterate from there. JIRA here: https://issues.apache.org/jira/browse/APEXMALHAR-2099 On 5/20/16, 1:06 PM, "Siyuan Hua" wrote: >Hi Ilya, > >I totally

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1 (Binding) Comments: Apache release tests failed on my machine with a multitude of errors related to @param notation. java version “1.8.0_77" OS X 10.11.4 PASSED: * Verified md5 and SHA512 for apache-apex-malhar-3.4.0-source-release.tar.gz * Verified md5 and SHA512 for apache-apex-malhar-3.4

Re: Getting various statitstics from StreamingContainers

I think it is appropriate to collect the information that the JVM provides using the available API instead of running external processes. For other information, how do you suggest that will be provided to the user? Thanks, Thomas On Mon, May 23, 2016 at 11:27 AM, Sandesh Hegde wrote: > Users

Re: Getting various statitstics from StreamingContainers

Users can pass the arguments to the JDK tools. So it exposes all the power of those tools. If we have to write the code we are doing the duplicate work. Also it doesn't evolve with the new features of the JVM, but the tools will and we just have to change the arguments that we pass. On Mon, May 23

Re: Getting various statitstics from StreamingContainers

What is the purpose of the new process? Why that information can't be collected directly from JVM and passed to app master using heartbeat? Thank you, Vlad On 5/23/16 10:57, Sandesh Hegde wrote: Hello All, Getting various information from the StreamingConatainers is a useful feature to have.

[jira] [Closed] (APEXCORE-468) Update verification page to remove check for disclaimer file

[ https://issues.apache.org/jira/browse/APEXCORE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise closed APEXCORE-468. - Resolution: Fixed > Update verification page to remove check for disclaimer file > -

[jira] [Commented] (APEXCORE-468) Update verification page to remove check for disclaimer file

[ https://issues.apache.org/jira/browse/APEXCORE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296782#comment-15296782 ] Thomas Weise commented on APEXCORE-468: --- Thanks! > Update verification page to r

[jira] [Commented] (APEXMALHAR-2094) Quantiles sketch operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296761#comment-15296761 ] Thomas Weise commented on APEXMALHAR-2094: -- Given your expansion plans, how

[GitHub] incubator-apex-site pull request: Updates in verification page to ...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-site/pull/39 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] incubator-apex-core pull request: *Review only* Prototype of runni...

Github user sandeshh commented on the pull request: https://github.com/apache/incubator-apex-core/pull/340#issuecomment-221046944 @tweise It is at the idea phase, I have started the discussion on the mailing thread. If this idea is fine then I will create a jira. --- If your project

Getting various statitstics from StreamingContainers

Hello All, Getting various information from the StreamingConatainers is a useful feature to have. As StreamingContainers are JVMs, various JDK tools can be used to get the information. So the idea is to spawn the new process from the streaming containers and return the information via Stram. Rec

[jira] [Commented] (APEXMALHAR-2094) Quantiles sketch operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296724#comment-15296724 ] Sandeep Narayanaswami commented on APEXMALHAR-2094: --- Thomas Weise T

[jira] [Issue Comment Deleted] (APEXMALHAR-2094) Quantiles sketch operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Narayanaswami updated APEXMALHAR-2094: -- Comment: was deleted (was: [~thw] Thomas, I do intend to add more f

[jira] [Commented] (APEXMALHAR-2094) Quantiles sketch operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296714#comment-15296714 ] Sandeep Narayanaswami commented on APEXMALHAR-2094: --- [~thw] Thomas,

[GitHub] incubator-apex-core pull request: *Review only* Prototype of runni...

Github user tweise commented on the pull request: https://github.com/apache/incubator-apex-core/pull/340#issuecomment-221041774 Any JIRA for this? It's recommended to discuss new features before submitting a PR to ensure they fit with other initiatives and avoid wasted effort.

[GitHub] incubator-apex-core pull request: *Review only* Prototype of runni...

GitHub user sandeshh opened a pull request: https://github.com/apache/incubator-apex-core/pull/340 *Review only* Prototype of running java utils to get information from containers I am using Jstack to get the container stack. But the idea is to run various commands ( jcmd, jinfo, j

[jira] [Updated] (APEXMALHAR-2099) Identify overlap between Beam API and existing Apex Stream API

[ https://issues.apache.org/jira/browse/APEXMALHAR-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated APEXMALHAR-2099: - Summary: Identify overlap between Beam API and existing Apex Stream API (was: Identi

[jira] [Created] (APEXMALHAR-2099) Identify overlap between Beam API and existing APex Stream API

Ilya Ganelin created APEXMALHAR-2099: Summary: Identify overlap between Beam API and existing APex Stream API Key: APEXMALHAR-2099 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2099 Projec

Re: Proposal for Hive output module

Sending it removing the links for better readability. Hi, Current implementation of HiveOperator is extended from AbstractStoreOutputOperator. We have an instance of store in AbstractStoreOutputOperator. Additionally, HiveOperator declares another field for store. I am thinking of removing this

[jira] [Updated] (APEXMALHAR-2075) Support fields of type Date,Time and Timestamp in Pojo Class for JdbcPOJOInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Deshmukh updated APEXMALHAR-2075: - Fix Version/s: 3.5.0 > Support fields of type Date,Time and Timestamp in Pojo

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1 Checked : 1) File Integrity Check. 2) Source Code Verification using tar.gz - Extracted tar file and compiled. - No binary files - Checked LICENSE, NOTICE, README, NOTICE. - rat check 3) Launched PI demo. Regards, Chaitanya On Mon, May 23, 2016 at 12:40 PM, Pradeep Kumbhar wr

[jira] [Commented] (APEXMALHAR-2075) Support fields of type Date,Time and Timestamp in Pojo Class for JdbcPOJOInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296204#comment-15296204 ] ASF GitHub Bot commented on APEXMALHAR-2075: Github user asfgit closed th

[GitHub] incubator-apex-malhar pull request: APEXMALHAR-2075 enhance jdbcin...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-apex-malhar/pull/269 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: Proposal for Hive output module

Hi, Current implementation of HiveOperator is extended from AbstractStoreOutputOperator

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1 Checked: -File integrity (gpg, md5sum, sha512sum): OK -Source code verification (using tar.gz): OK -Existence of LICENSE, NOTICE, README and CHANGELOG: OK -No unexpected binary files in the sources: OK -mvn rat check: OK -Launched PiDemo: OK On Mon, May 23, 2016 at 11:09 AM, Shubham Pathak