[jira] [Commented] (NIFI-5625) Support variables for the properties of HTTP processors
[ https://issues.apache.org/jira/browse/NIFI-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624966#comment-16624966 ] ASF GitHub Bot commented on NIFI-5625: -- GitHub user kemixkoo opened a pull request: https://github.com/apache/nifi/pull/3020 NIFI-5625: support the variables for the properties of HTTP processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kemixkoo/nifi NIFI-5625_support_expression Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3020.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3020 commit 46bc622eb79fc29573de496def0a0a7e49248933 Author: Kemix Koo Date: 2018-09-23T05:09:22Z NIFI-5625: support the variables for the properties of HTTP processors > Support variables for the properties of HTTP processors > > > Key: NIFI-5625 > URL: https://issues.apache.org/jira/browse/NIFI-5625 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions, Variable Registry >Affects Versions: 1.7.1 >Reporter: Kemix Koo >Priority: Minor > > When set some group (global) variables, some properties of HTTP processors > don't support the expressions. for example, USER and PASS, if don't set the > global variables, must change for each processors one by one. it's so > troublesomely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3020: NIFI-5625: support the variables for the properties...
GitHub user kemixkoo opened a pull request: https://github.com/apache/nifi/pull/3020 NIFI-5625: support the variables for the properties of HTTP processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kemixkoo/nifi NIFI-5625_support_expression Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3020.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3020 commit 46bc622eb79fc29573de496def0a0a7e49248933 Author: Kemix Koo Date: 2018-09-23T05:09:22Z NIFI-5625: support the variables for the properties of HTTP processors ---
[jira] [Created] (NIFI-5625) Support variables for the properties of HTTP processors
Kemix Koo created NIFI-5625: --- Summary: Support variables for the properties of HTTP processors Key: NIFI-5625 URL: https://issues.apache.org/jira/browse/NIFI-5625 Project: Apache NiFi Issue Type: Improvement Components: Extensions, Variable Registry Affects Versions: 1.7.1 Reporter: Kemix Koo When set some group (global) variables, some properties of HTTP processors don't support the expressions. for example, USER and PASS, if don't set the global variables, must change for each processors one by one. it's so troublesomely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2954: NIFI-5514: Fixed bugs in MergeRecord around minimum...
Github user bdesert commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2954#discussion_r219683471
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/MergeRecord.java
---
@@ -304,13 +336,25 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
session.commit();
}
+// If there is no more data queued up, complete any bin that meets
our minimum threshold
+int completedBins = 0;
+final QueueSize queueSize = session.getQueueSize();
--- End diff --
@markap14
Test for MinRecords is successful, but actual flow doesn't work as expected
and still doesn't respect Min Records setting.
There is my set up:
The problem is in different behavior of MockProcessSession and
StandardProcessSession.
MockProcessSession.getQueueSize() will return 0, after session.get(...)
StandardProcessSession.getQueueSize() will return same number as before
session.get(...), regardless flow files have been polled or not.
As a result, this condition will block from actually emitting FF when min
requirements are reached.
I would recommend to change this condition to:
if (flowFiles.size() != 0) {...}
In parallel, we probably need to open a JIRA for inconsistency between
MockProcessSession and StandardProcessSession.
---
[jira] [Commented] (NIFI-5514) MergeRecord does not create a merged FlowFile until a maximum threshold is reached
[
https://issues.apache.org/jira/browse/NIFI-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624850#comment-16624850
]
ASF GitHub Bot commented on NIFI-5514:
--
Github user bdesert commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2954#discussion_r219683471
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/MergeRecord.java
---
@@ -304,13 +336,25 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
session.commit();
}
+// If there is no more data queued up, complete any bin that meets
our minimum threshold
+int completedBins = 0;
+final QueueSize queueSize = session.getQueueSize();
--- End diff --
@markap14
Test for MinRecords is successful, but actual flow doesn't work as expected
and still doesn't respect Min Records setting.
There is my set up:
The problem is in different behavior of MockProcessSession and
StandardProcessSession.
MockProcessSession.getQueueSize() will return 0, after session.get(...)
StandardProcessSession.getQueueSize() will return same number as before
session.get(...), regardless flow files have been polled or not.
As a result, this condition will block from actually emitting FF when min
requirements are reached.
I would recommend to change this condition to:
if (flowFiles.size() != 0) {...}
In parallel, we probably need to open a JIRA for inconsistency between
MockProcessSession and StandardProcessSession.
> MergeRecord does not create a merged FlowFile until a maximum threshold is
> reached
> --
>
> Key: NIFI-5514
> URL: https://issues.apache.org/jira/browse/NIFI-5514
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> MergeRecord allows the user to specify a minimum number of records. However,
> if the minimum number of records is reached, the merged FlowFile is not
> created unless the maximum number of records or max number of bytes is also
> reached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2956 Rebased against master. ---
[jira] [Commented] (NIFI-5537) Create Neo4J cypher execution processor
[ https://issues.apache.org/jira/browse/NIFI-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624822#comment-16624822 ] ASF GitHub Bot commented on NIFI-5537: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2956 Rebased against master. > Create Neo4J cypher execution processor > --- > > Key: NIFI-5537 > URL: https://issues.apache.org/jira/browse/NIFI-5537 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.7.1 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: graph, neo4j, node, relationship > Fix For: 1.8.0 > > > Create Nifi Neo4J cypher queries processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5622) Test certificates require SAN values
[
https://issues.apache.org/jira/browse/NIFI-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624803#comment-16624803
]
ASF GitHub Bot commented on NIFI-5622:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/3018
I'm a +1 on this PR but will wait a bit in case someone with a security
background wants to double check.
> Test certificates require SAN values
>
>
> Key: NIFI-5622
> URL: https://issues.apache.org/jira/browse/NIFI-5622
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Tools and Build
>Affects Versions: 1.7.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: certificate, security, test
>
> During the update of OkHttp from 3.6.0 to 3.11.0 in
> [NIFI-4806|https://issues.apache.org/jira/browse/NIFI-4806], it was
> discovered that {{SubjectAlternativeName}} checking is now required, as
> described in RFC 6125. The test resource keystore and truststores need to be
> updates to provide SAN values.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #3018: NIFI-5622 Updated test resource keystores and truststores ...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3018 I'm a +1 on this PR but will wait a bit in case someone with a security background wants to double check. ---
[jira] [Commented] (NIFI-4731) BigQuery processors
[ https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624801#comment-16624801 ] ASF GitHub Bot commented on NIFI-4731: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3019 This PR supercedes #2682 and #2420. @ijokarumawak - would you mind having a look? (in particular in regards to the proxy settings) @zenfenan - a friendly ping since you looked at the one opened by @danieljimenez > BigQuery processors > --- > > Key: NIFI-4731 > URL: https://issues.apache.org/jira/browse/NIFI-4731 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mikhail Sosonkin >Priority: Major > > NIFI should have processors for putting data into BigQuery (Streaming and > Batch). > Initial working processors can be found this repository: > https://github.com/nologic/nifi/tree/NIFI-4731/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery > I'd like to get them into Nifi proper. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3019: [NIFI-4731][NIFI-4933] Big Query processor
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3019 This PR supercedes #2682 and #2420. @ijokarumawak - would you mind having a look? (in particular in regards to the proxy settings) @zenfenan - a friendly ping since you looked at the one opened by @danieljimenez ---
[jira] [Commented] (NIFI-4731) BigQuery processors
[ https://issues.apache.org/jira/browse/NIFI-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624799#comment-16624799 ] ASF GitHub Bot commented on NIFI-4731: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/3019 [NIFI-4731][NIFI-4933] Big Query processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi PR2682 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3019.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3019 commit e08d5cbd2b972bc92c39ccb5d2c6054ac2c5f852 Author: Daniel Jimenez Date: 2018-04-29T08:01:34Z NIFI-4731: BQ Processors and GCP library update. commit 6d55066fa6ac3e6adf60e566ecaa7f629def11a1 Author: Pierre Villard Date: 2018-09-18T21:19:03Z [NIFI-4731][NIFI-4933] BigQuery PR Review > BigQuery processors > --- > > Key: NIFI-4731 > URL: https://issues.apache.org/jira/browse/NIFI-4731 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mikhail Sosonkin >Priority: Major > > NIFI should have processors for putting data into BigQuery (Streaming and > Batch). > Initial working processors can be found this repository: > https://github.com/nologic/nifi/tree/NIFI-4731/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery > I'd like to get them into Nifi proper. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3019: [NIFI-4731][NIFI-4933] Big Query processor
GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/3019 [NIFI-4731][NIFI-4933] Big Query processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi PR2682 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3019.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3019 commit e08d5cbd2b972bc92c39ccb5d2c6054ac2c5f852 Author: Daniel Jimenez Date: 2018-04-29T08:01:34Z NIFI-4731: BQ Processors and GCP library update. commit 6d55066fa6ac3e6adf60e566ecaa7f629def11a1 Author: Pierre Villard Date: 2018-09-18T21:19:03Z [NIFI-4731][NIFI-4933] BigQuery PR Review ---
