[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99126037 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); +if (StringUtils.isNotBlank(customHeaderValue.getValue())) { --- End diff -- Yep, following. Will give this one more look through. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99123631 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); --- End diff -- You would be 100% correct. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99039594 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + --- End diff -- fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99039524 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; --- End diff -- @alopresto I saw that one before... Challenge is trying not to forget doing it upon setting up a new dev environment. :-) Damn you memory for forgetting you would damn intellij for optimising imports. :-D --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99039115 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; --- End diff -- You can set this in **Preferences > Editor > Code Style > Java**, *Imports* tab and then set the following properties: * Class count to use import with '*': `999` * Names count to use static import with '*': `999` You can also import the [Apache NiFi Checkstyle settings](https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-IntelliJIDEAUsers) into **Other > Checkstyle**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038955 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + +"whenever making a new HTTP request. \n " + +"Setting this the name to XYZ and value to ${attribute} will result on the header 'XYZ: attribute_value' being sent to the HTTP endpoint"), --- End diff -- Minor things! I am English as an Only Language, so you are totally winning in this area! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038769 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + --- End diff -- Maybe just "Name of Header"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038757 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); +if (StringUtils.isNotBlank(customHeaderValue.getValue())) { --- End diff -- I was working under two assumptions: * onScheduled is called once per scheduling; * in theory use time related expression language even if incoming flowfiles were to be absent Does it make sense? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038321 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); --- End diff -- I considered that but refrained from it due to: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java#L99 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038165 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + +"whenever making a new HTTP request. \n " + +"Setting this the name to XYZ and value to ${attribute} will result on the header 'XYZ: attribute_value' being sent to the HTTP endpoint"), --- End diff -- Oh ESL... :-) Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038126 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + --- End diff -- I agree but couldn't think of a better way of conveying the message. Happy to give it another go. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038013 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; --- End diff -- Damn you intellij!!! :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99038019 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; -import org.apache.nifi.annotation.behavior.Stateful; -import org.apache.nifi.annotation.behavior.WritesAttribute; -import org.apache.nifi.annotation.behavior.WritesAttributes; import org.apache.nifi.annotation.documentation.CapabilityDescription; import org.apache.nifi.annotation.documentation.Tags; import org.apache.nifi.annotation.lifecycle.OnScheduled; -import org.apache.nifi.components.AllowableValue; -import org.apache.nifi.components.PropertyDescriptor; -import org.apache.nifi.components.ValidationContext; -import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.*; --- End diff -- Damn you intellij!!! :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99034248 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; --- End diff -- Should avoid the * imports. This didn't seem to fail contrib-check for me, but is a rule in the checkstyle config --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99034256 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -62,18 +62,12 @@ import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.impl.conn.BasicHttpClientConnectionManager; import org.apache.http.ssl.SSLContextBuilder; -import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.*; import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; -import org.apache.nifi.annotation.behavior.Stateful; -import org.apache.nifi.annotation.behavior.WritesAttribute; -import org.apache.nifi.annotation.behavior.WritesAttributes; import org.apache.nifi.annotation.documentation.CapabilityDescription; import org.apache.nifi.annotation.documentation.Tags; import org.apache.nifi.annotation.lifecycle.OnScheduled; -import org.apache.nifi.components.AllowableValue; -import org.apache.nifi.components.PropertyDescriptor; -import org.apache.nifi.components.ValidationContext; -import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.components.*; --- End diff -- Should avoid the * imports. This didn't seem to fail contrib-check for me, but is a rule in the checkstyle config --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99036818 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); --- End diff -- I could see this potentially making use of the incoming FlowFile as seed data for the EL. Perhaps we should pass in the flowFile? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99034506 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + +"whenever making a new HTTP request. \n " + +"Setting this the name to XYZ and value to ${attribute} will result on the header 'XYZ: attribute_value' being sent to the HTTP endpoint"), --- End diff -- "Setting this the" -> "Setting this to the name" "on" -> "in" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99034384 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -102,6 +96,11 @@ + "once the content has been fetched from the given URL, it will not be fetched again until the content on the remote server changes. Note that due to limitations on state " + "management, stored \"last modified\" and etag fields never expire. If the URL in GetHttp uses Expression Language that is unbounded, there " + "is the potential for Out of Memory Errors to occur.") +@DynamicProperties({ +@DynamicProperty(name = "header_XYZ", value = "The Expression Language to be used to populate the header value", description = "The additional headers to be sent by the processor " + --- End diff -- This is a little confusing as it seems like the prefix header_ is needed before the name of the actual header but that doesn't seem to be the implementation. What are your thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1462: NIFI-819 - Extend GetHTTP to use dynamically add HT...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1462#discussion_r99035247 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java --- @@ -467,6 +486,18 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory if (accept != null) { get.addHeader(HEADER_ACCEPT, accept); } + +// Add dynamic headers + +PropertyValue customHeaderValue; +for (PropertyDescriptor customProperty : customHeaders) { +customHeaderValue = context.getProperty(customProperty).evaluateAttributeExpressions(); +if (StringUtils.isNotBlank(customHeaderValue.getValue())) { --- End diff -- I think it might make more sense to do this in onScheduled, although depending on the decision made with the evaluation of attribute expressions, above, may be fine as it currently stands. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---