[jira] [Updated] (NIFI-4979) Fix ReportLineageToAtlas documentation errors
[ https://issues.apache.org/jira/browse/NIFI-4979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4979: Status: Patch Available (was: In Progress) > Fix ReportLineageToAtlas documentation errors > - > > Key: NIFI-4979 > URL: https://issues.apache.org/jira/browse/NIFI-4979 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Following docs should be updated: > * CapabilityDescription, sentence is not finished correctly > * "NiFi Lineage Strategy" should have more note on the effects of switching > strategy > * Remote Input/Output Port descriptions in Additional Details "Supported > DataSets and Processors" table are not accurate > * The note on Atlas version for kafka_topic In the same table is not correct -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4979) Fix ReportLineageToAtlas documentation errors
[ https://issues.apache.org/jira/browse/NIFI-4979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399861#comment-16399861 ] ASF GitHub Bot commented on NIFI-4979: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2549 NIFI-4979: Fix ReportLineageToAtlas documentation errors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4979 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2549.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2549 commit e68e7cf7279811c693c40c382680281c4282ed9e Author: Koji Kawamura Date: 2018-03-15T03:22:31Z NIFI-4979: Fix ReportLineageToAtlas documentation errors > Fix ReportLineageToAtlas documentation errors > - > > Key: NIFI-4979 > URL: https://issues.apache.org/jira/browse/NIFI-4979 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Following docs should be updated: > * CapabilityDescription, sentence is not finished correctly > * "NiFi Lineage Strategy" should have more note on the effects of switching > strategy > * Remote Input/Output Port descriptions in Additional Details "Supported > DataSets and Processors" table are not accurate > * The note on Atlas version for kafka_topic In the same table is not correct -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2549: NIFI-4979: Fix ReportLineageToAtlas documentation e...
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2549 NIFI-4979: Fix ReportLineageToAtlas documentation errors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4979 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2549.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2549 commit e68e7cf7279811c693c40c382680281c4282ed9e Author: Koji Kawamura Date: 2018-03-15T03:22:31Z NIFI-4979: Fix ReportLineageToAtlas documentation errors ---
[jira] [Created] (NIFI-4979) Fix ReportLineageToAtlas documentation errors
Koji Kawamura created NIFI-4979: --- Summary: Fix ReportLineageToAtlas documentation errors Key: NIFI-4979 URL: https://issues.apache.org/jira/browse/NIFI-4979 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.5.0 Reporter: Koji Kawamura Assignee: Koji Kawamura Following docs should be updated: * CapabilityDescription, sentence is not finished correctly * "NiFi Lineage Strategy" should have more note on the effects of switching strategy * Remote Input/Output Port descriptions in Additional Details "Supported DataSets and Processors" table are not accurate * The note on Atlas version for kafka_topic In the same table is not correct -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant
[ https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399767#comment-16399767 ] ASF GitHub Bot commented on NIFI-4246: -- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174658418 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws I
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174658418 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws InitializationException if unable to create a database connection + */ +@OnEnabled +public void onEnabled(final ConfigurationContext context) throws InitializationException { + +super.onEnabled(context); +
[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant
[ https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399765#comment-16399765 ] ASF GitHub Bot commented on NIFI-4246: -- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174658141 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws I
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174658141 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws InitializationException if unable to create a database connection + */ +@OnEnabled +public void onEnabled(final ConfigurationContext context) throws InitializationException { + +super.onEnabled(context); +
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174656145 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws InitializationException if unable to create a database connection + */ +@OnEnabled +public void onEnabled(final ConfigurationContext context) throws InitializationException { + +super.onEnabled(context); +
[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant
[ https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399737#comment-16399737 ] ASF GitHub Bot commented on NIFI-4246: -- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/2085#discussion_r174656145 --- Diff: nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java --- @@ -0,0 +1,167 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Created on 7/25/17. + */ + +package org.apache.nifi.oauth; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.reporting.InitializationException; +import org.apache.oltu.oauth2.client.HttpClient; +import org.apache.oltu.oauth2.client.request.OAuthClientRequest; +import org.apache.oltu.oauth2.common.exception.OAuthProblemException; +import org.apache.oltu.oauth2.common.exception.OAuthSystemException; +import org.apache.oltu.oauth2.common.message.types.GrantType; + +@Tags({ "oauth2", "client", "secret", "post"}) +@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2 authentication server to retrieve the" + +" access token. The access token is stored locally in the controller service and used by processors " + +"referencing this controller service.") +public class OAuth2ClientCredentialsGrantControllerService +extends AbstractOAuthControllerService +implements OAuth2ClientService { + +public static final PropertyDescriptor CLIENT_ID = new PropertyDescriptor +.Builder().name("OAuth2 Client ID") +.displayName("OAuth2 Client ID") +.description("OAuth2 Client ID passed to the authorization server") +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor CLIENT_SECRET = new PropertyDescriptor +.Builder().name("OAuth2 Client Secret") +.displayName("OAuth2 Client Secret") +.description("OAuth2 Client Secret that will be passed to the authorization server in exchange for an access token") +.sensitive(true) +.required(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +private static final List properties; + +static { +final List props = new ArrayList<>(); +props.add(AUTH_SERVER_URL); +props.add(CLIENT_ID); +props.add(CLIENT_SECRET); +props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME); +props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME); +props.add(RESPONSE_EXPIRE_IN_FIELD_NAME); +props.add(RESPONSE_SCOPE_FIELD_NAME); +props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME); +properties = Collections.unmodifiableList(props); +} + +@Override +protected List getSupportedPropertyDescriptors() { +return properties; +} + +private String clientId = null; +private String clientSecret = null; +private Base64.Encoder enc = Base64.getEncoder(); + +/** + * @param context the configuration context + * @throws I
[jira] [Updated] (NIFI-4978) Stopping of ReportLineageToAtlas can cause NPE
[ https://issues.apache.org/jira/browse/NIFI-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4978: Status: Patch Available (was: In Progress) > Stopping of ReportLineageToAtlas can cause NPE > -- > > Key: NIFI-4978 > URL: https://issues.apache.org/jira/browse/NIFI-4978 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Stopping ReportLineageToAtlas can cause following NPE. > {code:java} > 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] > org.apache.atlas.AtlasBaseClient Client has only one service URL, will use > that for all actions: > http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 > 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run > 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = > false > 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] > o.a.n.a.reporting.ReportLineageToAtlas > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running > task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to > java.lang.NullPointerException > 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] > o.a.n.a.reporting.ReportLineageToAtlas > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : > java.lang.NullPointerException > java.lang.NullPointerException: null > at > org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) > at > org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu > {code} > This issue was reported by [~nayakmahesh616] > The NPE happened because nifiAtlasHook is null cleared by @OnUnscheduled > while onTrigger method is still running. That cleanup code should be executed > by @OnStopped. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4978) Stopping of ReportLineageToAtlas can cause NPE
[ https://issues.apache.org/jira/browse/NIFI-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4978: Description: Stopping ReportLineageToAtlas can cause following NPE. {code:java} 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] org.apache.atlas.AtlasBaseClient Client has only one service URL, will use that for all actions: http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = false 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to java.lang.NullPointerException 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu {code} This issue was reported by [~nayakmahesh616] The NPE happened because nifiAtlasHook is null cleared by @OnUnscheduled while onTrigger method is still running. That cleanup code should be executed by @OnStopped. was: Stopping ReportLineageToAtlas can cause following NPE. {code} 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] org.apache.atlas.AtlasBaseClient Client has only one service URL, will use that for all actions: http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = false 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to java.lang.NullPointerException 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu {code} This issue was reported by [~nayakmahesh616] > Stopping of ReportLineageToAtlas can cause NPE > -- > > Key: NIFI-4978 > URL: https://issues.apache.org/jira/browse/NIFI-4978 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Stopping ReportLineageToAtlas can cause following NPE. > {code:java} > 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] > org.apache.atlas.AtlasBaseClient Client has only one service URL, will use > that for all actions: > http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 > 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run > 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2a3e0c4a // Another
[jira] [Commented] (NIFI-4978) Stopping of ReportLineageToAtlas can cause NPE
[ https://issues.apache.org/jira/browse/NIFI-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399705#comment-16399705 ] ASF GitHub Bot commented on NIFI-4978: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2548 NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4978 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2548.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2548 commit 31bfbc1f8a39d45884f599adb4de33899ff31e67 Author: Koji Kawamura Date: 2018-03-14T19:22:26Z NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled > Stopping of ReportLineageToAtlas can cause NPE > -- > > Key: NIFI-4978 > URL: https://issues.apache.org/jira/browse/NIFI-4978 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Stopping ReportLineageToAtlas can cause following NPE. > {code} > 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] > org.apache.atlas.AtlasBaseClient Client has only one service URL, will use > that for all actions: > http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 > 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run > 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = > false > 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] > o.a.n.a.reporting.ReportLineageToAtlas > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running > task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to > java.lang.NullPointerException > 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] > o.a.n.a.reporting.ReportLineageToAtlas > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : > java.lang.NullPointerException > java.lang.NullPointerException: null > at > org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) > at > org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu > {code} > This issue was reported
[GitHub] nifi pull request #2548: NIFI-4978: Fixed ReportLineageToAtlas NPE when unsc...
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2548 NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4978 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2548.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2548 commit 31bfbc1f8a39d45884f599adb4de33899ff31e67 Author: Koji Kawamura Date: 2018-03-14T19:22:26Z NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled ---
[jira] [Updated] (NIFI-4978) Stopping of ReportLineageToAtlas can cause NPE
[ https://issues.apache.org/jira/browse/NIFI-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4978: Description: Stopping ReportLineageToAtlas can cause following NPE. {code} 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] org.apache.atlas.AtlasBaseClient Client has only one service URL, will use that for all actions: http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = false 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to java.lang.NullPointerException 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu {code} This issue was reported by [~nayakmahesh616] was: Stopping ReportLineageToAtlas can cause following NPE. {code} 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] org.apache.atlas.AtlasBaseClient Client has only one service URL, will use that for all actions: http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = false 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to java.lang.NullPointerException 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu {code} > Stopping of ReportLineageToAtlas can cause NPE > -- > > Key: NIFI-4978 > URL: https://issues.apache.org/jira/browse/NIFI-4978 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Minor > > Stopping ReportLineageToAtlas can cause following NPE. > {code} > 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] > org.apache.atlas.AtlasBaseClient Client has only one service URL, will use > that for all actions: > http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 > 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run > 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = > false > 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] > o.a.n.a.reporting.ReportLineageToAtlas > ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running > task Report
[jira] [Created] (NIFI-4978) Stopping of ReportLineageToAtlas can cause NPE
Koji Kawamura created NIFI-4978: --- Summary: Stopping of ReportLineageToAtlas can cause NPE Key: NIFI-4978 URL: https://issues.apache.org/jira/browse/NIFI-4978 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.5.0 Reporter: Koji Kawamura Assignee: Koji Kawamura Stopping ReportLineageToAtlas can cause following NPE. {code} 2018-02-21 11:34:16,490 INFO [Timer-Driven Process Thread-8] org.apache.atlas.AtlasBaseClient Client has only one service URL, will use that for all actions: http://ctr-e138-1518143905142-29976-01-02.hwx.site:21000 2018-02-21 11:34:16,711 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] to run 2018-02-21 11:34:16,760 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2a3e0c4a // Another save pending = false 2018-02-21 11:34:17,110 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] Error running task ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] due to java.lang.NullPointerException 2018-02-21 11:34:17,111 ERROR [Timer-Driven Process Thread-8] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=b7bf3781-0161-1000--afd9ddbd] : java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:614) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecu {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4977) PutSyslog should support Expression Language for it's Sender properties
[ https://issues.apache.org/jira/browse/NIFI-4977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-4977: --- Status: Patch Available (was: Open) > PutSyslog should support Expression Language for it's Sender properties > --- > > Key: NIFI-4977 > URL: https://issues.apache.org/jira/browse/NIFI-4977 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > > As part of the continued effort to add access to the variable registry to > processors, the PutSyslog processor should expose expression language to it's > sender properties. These properties control where and how the processor sends > to. > This update should better align PutSyslog with the other PutX processors > which support EL on the sender properties (Slack, TCP, UDP). > These properties are not evaluated per FlowFile and shouldn't be expected to > change between onTriggers (since the sender pool is shared) so it will need > to be documented as such that the -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4977) PutSyslog should support Expression Language for it's Sender properties
[ https://issues.apache.org/jira/browse/NIFI-4977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399678#comment-16399678 ] ASF GitHub Bot commented on NIFI-4977: -- GitHub user JPercivall opened a pull request: https://github.com/apache/nifi/pull/2547 NIFI-4977 Adding expression language support to the Sender properties… … of PutSyslog Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JPercivall/nifi NIFI-4977_Exposing_EL_in_PutSyslog Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2547.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2547 commit e9b731529a243ec439760a758cb0c809f7f8cd93 Author: Joe Percivall Date: 2018-03-15T00:39:04Z NIFI-4977 Adding expression language support to the Sender properties of PutSyslog > PutSyslog should support Expression Language for it's Sender properties > --- > > Key: NIFI-4977 > URL: https://issues.apache.org/jira/browse/NIFI-4977 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > > As part of the continued effort to add access to the variable registry to > processors, the PutSyslog processor should expose expression language to it's > sender properties. These properties control where and how the processor sends > to. > This update should better align PutSyslog with the other PutX processors > which support EL on the sender properties (Slack, TCP, UDP). > These properties are not evaluated per FlowFile and shouldn't be expected to > change between onTriggers (since the sender pool is shared) so it will need > to be documented as such that the -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2547: NIFI-4977 Adding expression language support to the...
GitHub user JPercivall opened a pull request: https://github.com/apache/nifi/pull/2547 NIFI-4977 Adding expression language support to the Sender properties⦠⦠of PutSyslog Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JPercivall/nifi NIFI-4977_Exposing_EL_in_PutSyslog Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2547.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2547 commit e9b731529a243ec439760a758cb0c809f7f8cd93 Author: Joe Percivall Date: 2018-03-15T00:39:04Z NIFI-4977 Adding expression language support to the Sender properties of PutSyslog ---
[jira] [Commented] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site
[ https://issues.apache.org/jira/browse/MINIFICPP-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399656#comment-16399656 ] ASF GitHub Bot commented on MINIFICPP-404: -- GitHub user minifirocks opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/280 MINIFICPP-404: http proxy support for s2s Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/minifirocks/nifi-minifi-cpp http_proxy Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/280.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #280 commit e3ae4cdae5e1b474a4bab0b8cb5fe530f6a7f741 Author: Bin Qiu Date: 2018-03-08T16:05:35Z MINIFICPP-404: http proxy support for s2s > HTTP Proxy Support for HTTP Site to Site > > > Key: MINIFICPP-404 > URL: https://issues.apache.org/jira/browse/MINIFICPP-404 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > HTTP Proxy Support for HTTP Site to Site > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > support for this in YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #280: MINIFICPP-404: http proxy support for s2s
GitHub user minifirocks opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/280 MINIFICPP-404: http proxy support for s2s Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/minifirocks/nifi-minifi-cpp http_proxy Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/280.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #280 commit e3ae4cdae5e1b474a4bab0b8cb5fe530f6a7f741 Author: Bin Qiu Date: 2018-03-08T16:05:35Z MINIFICPP-404: http proxy support for s2s ---
[jira] [Created] (NIFI-4977) PutSyslog should support Expression Language for it's Sender properties
Joseph Percivall created NIFI-4977: -- Summary: PutSyslog should support Expression Language for it's Sender properties Key: NIFI-4977 URL: https://issues.apache.org/jira/browse/NIFI-4977 Project: Apache NiFi Issue Type: Improvement Reporter: Joseph Percivall Assignee: Joseph Percivall As part of the continued effort to add access to the variable registry to processors, the PutSyslog processor should expose expression language to it's sender properties. These properties control where and how the processor sends to. This update should better align PutSyslog with the other PutX processors which support EL on the sender properties (Slack, TCP, UDP). These properties are not evaluated per FlowFile and shouldn't be expected to change between onTriggers (since the sender pool is shared) so it will need to be documented as such that the -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
[ https://issues.apache.org/jira/browse/NIFI-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399489#comment-16399489 ] ASF GitHub Bot commented on NIFI-4976: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2545 > ConsumeJMS occasionally failing when reading TextMessage from IBM MQ > > > Key: NIFI-4976 > URL: https://issues.apache.org/jira/browse/NIFI-4976 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When reading from IBM MQ we will occasionally encounter a stack trace like > the following: > {code:java} > 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: > Failed to convert BytesMessage to byte[] > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) > at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) > at > org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) > at > org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) > at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The > character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: > 63' cannot convert some or all of the string '[B@14c194bd' > at sun.reflect.GeneratedConstructorAccessor1012.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) > at > com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421) > at > com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240) > at > com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205) > at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124) > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40) > ... 19 common frames omitted > Caused by: java.nio.charset.MalformedInputException: Input length = 1 > at java.nio.charset.CoderResult.throwException(CoderResult.java:281) > at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) > at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375) > ... 24 common frames omitted{code} > Unfortunately, if using Client Acknowledge, we end up not acknowledging the > message, since we can't read it, so it stays on the queue, and we constantly > attempt to pull it again, causing the processor to essentially become 'stuck'. > If such an issue arises, where we cannot retrieve the content of
[jira] [Resolved] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
[ https://issues.apache.org/jira/browse/NIFI-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-4976. -- Resolution: Fixed Fix Version/s: 1.6.0 > ConsumeJMS occasionally failing when reading TextMessage from IBM MQ > > > Key: NIFI-4976 > URL: https://issues.apache.org/jira/browse/NIFI-4976 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When reading from IBM MQ we will occasionally encounter a stack trace like > the following: > {code:java} > 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: > Failed to convert BytesMessage to byte[] > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) > at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) > at > org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) > at > org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) > at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The > character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: > 63' cannot convert some or all of the string '[B@14c194bd' > at sun.reflect.GeneratedConstructorAccessor1012.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) > at > com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421) > at > com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240) > at > com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205) > at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124) > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40) > ... 19 common frames omitted > Caused by: java.nio.charset.MalformedInputException: Input length = 1 > at java.nio.charset.CoderResult.throwException(CoderResult.java:281) > at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) > at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375) > ... 24 common frames omitted{code} > Unfortunately, if using Client Acknowledge, we end up not acknowledging the > message, since we can't read it, so it stays on the queue, and we constantly > attempt to pull it again, causing the processor to essentially become 'stuck'. > If such an issue arises, where we cannot retrieve the content of the message, > we are better off to log an ERROR with the stack trace and then acknowledge > the m
[jira] [Commented] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
[ https://issues.apache.org/jira/browse/NIFI-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399487#comment-16399487 ] ASF subversion and git services commented on NIFI-4976: --- Commit 867ffdb52edac5cd616f9f05b48bbb76a8c5cfa9 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=867ffdb ] NIFI-4976: If unable to retrieve message content, warn an error but acknowledge message. Signed-off-by: Pierre Villard This closes #2545. > ConsumeJMS occasionally failing when reading TextMessage from IBM MQ > > > Key: NIFI-4976 > URL: https://issues.apache.org/jira/browse/NIFI-4976 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When reading from IBM MQ we will occasionally encounter a stack trace like > the following: > {code:java} > 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: > Failed to convert BytesMessage to byte[] > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) > at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) > at > org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) > at > org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) > at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The > character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: > 63' cannot convert some or all of the string '[B@14c194bd' > at sun.reflect.GeneratedConstructorAccessor1012.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) > at > com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421) > at > com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240) > at > com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205) > at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124) > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40) > ... 19 common frames omitted > Caused by: java.nio.charset.MalformedInputException: Input length = 1 > at java.nio.charset.CoderResult.throwException(CoderResult.java:281) > at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) > at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375) > ... 24 common frames omitted{code} > Unfortunately, if using Client Acknowledge, we end up not acknowledging the > message
[GitHub] nifi pull request #2545: NIFI-4976: If unable to retrieve message content, w...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2545 ---
[jira] [Commented] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
[ https://issues.apache.org/jira/browse/NIFI-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399486#comment-16399486 ] ASF GitHub Bot commented on NIFI-4976: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2545 LGTM, merging to master, thanks @markap14 > ConsumeJMS occasionally failing when reading TextMessage from IBM MQ > > > Key: NIFI-4976 > URL: https://issues.apache.org/jira/browse/NIFI-4976 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When reading from IBM MQ we will occasionally encounter a stack trace like > the following: > {code:java} > 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: > Failed to convert BytesMessage to byte[] > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) > at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) > at > org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) > at > org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) > at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The > character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: > 63' cannot convert some or all of the string '[B@14c194bd' > at sun.reflect.GeneratedConstructorAccessor1012.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) > at > com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421) > at > com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240) > at > com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205) > at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124) > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40) > ... 19 common frames omitted > Caused by: java.nio.charset.MalformedInputException: Input length = 1 > at java.nio.charset.CoderResult.throwException(CoderResult.java:281) > at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) > at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690) > at > com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375) > ... 24 common frames omitted{code} > Unfortunately, if using Client Acknowledge, we end up not acknowledging the > message, since we can't read it, so it stays on the queue, and we constantly > attempt to pull it again, causing the processor to essentially become 'stuck'. > If such an issue arises, where we cannot retri
[GitHub] nifi issue #2545: NIFI-4976: If unable to retrieve message content, warn an ...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2545 LGTM, merging to master, thanks @markap14 ---
[jira] [Commented] (NIFI-4882) CSVRecordReader should utilize specified date/time/timestamp format at its convertSimpleIfPossible method
[ https://issues.apache.org/jira/browse/NIFI-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399361#comment-16399361 ] ASF GitHub Bot commented on NIFI-4882: -- Github user derekstraka commented on the issue: https://github.com/apache/nifi/pull/2473 @ijokarumawak - Any update on this? I can rebase once you think it is good to go in. thanks! > CSVRecordReader should utilize specified date/time/timestamp format at its > convertSimpleIfPossible method > - > > Key: NIFI-4882 > URL: https://issues.apache.org/jira/browse/NIFI-4882 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Koji Kawamura >Assignee: Derek Straka >Priority: Major > > CSVRecordReader.convertSimpleIfPossible method is used by ValidateRecord. The > method does not coerce values to the target schema field type if the raw > string representation in the input CSV file is not compatible. > The type compatibility check is implemented as follows. But it does not use > user specified date/time/timestamp format: > {code} > // This will return 'false' for input '01/01/1900' when user > specified custom format 'MM/dd/' > if (DataTypeUtils.isCompatibleDataType(trimmed, dataType)) { > // The LAZY_DATE_FORMAT should be used to check > compatibility, too. > return DataTypeUtils.convertType(trimmed, dataType, > LAZY_DATE_FORMAT, LAZY_TIME_FORMAT, LAZY_TIMESTAMP_FORMAT, fieldName); > } else { > return value; > } > {code} > If input date strings have different format than the default format > '-MM-dd', then ValidateRecord processor can not validate input records. > JacksonCSVRecordReader has the identical methods with CSVRecordReader. Those > classes should have an abstract class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2473: NIFI-4882: Resolve issue with parsing custom date, time, a...
Github user derekstraka commented on the issue: https://github.com/apache/nifi/pull/2473 @ijokarumawak - Any update on this? I can rebase once you think it is good to go in. thanks! ---
[GitHub] nifi issue #2085: NIFI-4246 - Client Credentials Grant based OAuth2 Controll...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2085 There are merge conflicts in `InvokeHTTP` and in a number of the poms. I believe I have resolved all of these [6a80063313f9c8e03a1f42ae84813cabd5fc8349](https://github.com/alopresto/nifi/commit/6a80063313f9c8e03a1f42ae84813cabd5fc8349). There are also the following issues: * It looks like somehow an empty file was added at `nifi/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/webapp/WEB-INF/jsp/authorize.jsp`. I only noticed this because it failed the RAT check. I imagine this can be deleted? * An unused field remains in `AbstractOAuthControllerService`: `protected long expireTimeSafetyNetSeconds = -1;` * `OAuth2ClientCredentialsGrantControllerService.CLIENT_ID` and `OAuth2ClientCredentialsGrantControllerService.CLIENT_SECRET` don't support expression language (with the variable registry and NiFi Registry, these values will likely be populated externally) * The exception handling in `OAuth2ClientCredentialsGrantControllerService#authenticate` can be condensed * There are no unit or integration tests. I understand a real integration test with an external service is difficult, but unit tests for `OAuth2ClientCredentialsGrantControllerService#authenticate()` (via mocking `OAuthHTTPConnectionClient#execute()`) and `AbstractOAuthControllerService#isOAuthTokenExpired()` would be helpful * There is no update to the User Guide or Admin Guide describing this feature. At a minimum, the `InvokeHTTP` processor documentation should be updated I am in the process of configuring an external OAuth service to use with this. Will have a final review once that evaluation is complete. ---
[jira] [Commented] (NIFI-4246) OAuth 2 Authorization support - Client Credentials Grant
[ https://issues.apache.org/jira/browse/NIFI-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399354#comment-16399354 ] ASF GitHub Bot commented on NIFI-4246: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2085 There are merge conflicts in `InvokeHTTP` and in a number of the poms. I believe I have resolved all of these [6a80063313f9c8e03a1f42ae84813cabd5fc8349](https://github.com/alopresto/nifi/commit/6a80063313f9c8e03a1f42ae84813cabd5fc8349). There are also the following issues: * It looks like somehow an empty file was added at `nifi/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-ui/src/main/webapp/WEB-INF/jsp/authorize.jsp`. I only noticed this because it failed the RAT check. I imagine this can be deleted? * An unused field remains in `AbstractOAuthControllerService`: `protected long expireTimeSafetyNetSeconds = -1;` * `OAuth2ClientCredentialsGrantControllerService.CLIENT_ID` and `OAuth2ClientCredentialsGrantControllerService.CLIENT_SECRET` don't support expression language (with the variable registry and NiFi Registry, these values will likely be populated externally) * The exception handling in `OAuth2ClientCredentialsGrantControllerService#authenticate` can be condensed * There are no unit or integration tests. I understand a real integration test with an external service is difficult, but unit tests for `OAuth2ClientCredentialsGrantControllerService#authenticate()` (via mocking `OAuthHTTPConnectionClient#execute()`) and `AbstractOAuthControllerService#isOAuthTokenExpired()` would be helpful * There is no update to the User Guide or Admin Guide describing this feature. At a minimum, the `InvokeHTTP` processor documentation should be updated I am in the process of configuring an external OAuth service to use with this. Will have a final review once that evaluation is complete. > OAuth 2 Authorization support - Client Credentials Grant > > > Key: NIFI-4246 > URL: https://issues.apache.org/jira/browse/NIFI-4246 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > If your interacting with REST endpoints on the web chances are you are going > to run into an OAuth2 secured webservice. The IETF (Internet Engineering Task > Force) defines 4 methods in which OAuth2 authorization can occur. This JIRA > is focused solely on the Client Credentials Grant method defined at > https://tools.ietf.org/html/rfc6749#section-4.4 > This implementation should provide a ControllerService in which the enduser > can configure the credentials for obtaining the authorization grant (access > token) from the resource owner. In turn a new property will be added to the > InvokeHTTP processor (if it doesn't already exist from one of the other JIRA > efforts similar to this one) where the processor can reference this > controller service to obtain the access token and insert the appropriate HTTP > header (Authorization: Bearer{access_token}) so that the InvokeHTTP processor > can interact with the OAuth protected resources without having to worry about > setting up the credentials for each InvokeHTTP processor saving time and > complexity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-4973) RPG port merging
[ https://issues.apache.org/jira/browse/NIFI-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4973: - Assignee: Matt Gilman > RPG port merging > > > Key: NIFI-4973 > URL: https://issues.apache.org/jira/browse/NIFI-4973 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > > When clustered, the available ports that should be returned to the client > should only include ports that all nodes have discovered. It appears that > currently, it's returning the union and not the intersection. The code in > question is in RemoteProcessGroupEntityMerger#mergeDtos(...) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFIREG-152) Support storing flows in Object Store/S3
Daniel Oakley created NIFIREG-152: - Summary: Support storing flows in Object Store/S3 Key: NIFIREG-152 URL: https://issues.apache.org/jira/browse/NIFIREG-152 Project: NiFi Registry Issue Type: Improvement Affects Versions: 0.2.0 Reporter: Daniel Oakley It would be nice if the registry had an option to store saved flow versions as immutable objects in an S3 object store or similar. It would mean less local storage configuration needed which is useful when running under Docker/Kubernetes for example. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-405) RPG Binding to Local Network Interface
[ https://issues.apache.org/jira/browse/MINIFICPP-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu resolved MINIFICPP-405. Resolution: Fixed > RPG Binding to Local Network Interface > -- > > Key: MINIFICPP-405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-405 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Remote Process Group for site2site bind to a local interface name > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2507: Nifi 3093
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2507 I'm leery of adding "Hive support" when we know there's a defect, especially when we had to fix that processor for the exact same reason (other folks were running into issues using GenerateTableFetch once it accepted multiple tables). There's also a separate thread on the mailing lists about adding the database name to the table + column name, and I think it's probably a good idea. So we wouldn't be able to support an empty table name if we add a database name, and it's a bummer to default to an empty table name when we do know how to get the table name for the Hive case. ---
[GitHub] nifi pull request #2468: NIFI-4849: Implemented REST Endpoint and associated...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174577132 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/diagnostics/RepositoryUsageDTO.java --- @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.web.api.dto.diagnostics; + +import javax.xml.bind.annotation.XmlType; + +import io.swagger.annotations.ApiModelProperty; + +@XmlType(name = "fileStoreUsage") --- End diff -- Good catch -- renamed the class and forgot to rename the XmlType ---
[jira] [Commented] (NIFI-4849) Add REST Endpoint for gathering Processor Diagnostics information
[ https://issues.apache.org/jira/browse/NIFI-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399129#comment-16399129 ] ASF GitHub Bot commented on NIFI-4849: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174577132 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/diagnostics/RepositoryUsageDTO.java --- @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.web.api.dto.diagnostics; + +import javax.xml.bind.annotation.XmlType; + +import io.swagger.annotations.ApiModelProperty; + +@XmlType(name = "fileStoreUsage") --- End diff -- Good catch -- renamed the class and forgot to rename the XmlType > Add REST Endpoint for gathering Processor Diagnostics information > - > > Key: NIFI-4849 > URL: https://issues.apache.org/jira/browse/NIFI-4849 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > We need to add a REST endpoint that will use the appropriate resources to > gather the Processor Diagnostics information. Information to return should > include things like: > * Processor config > * Processor status > * Garbage Collection info > * Repo Sizes > * Connection info for connections whose source or destination is the > processor > * Controller Services that the processor is referencing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4849) Add REST Endpoint for gathering Processor Diagnostics information
[ https://issues.apache.org/jira/browse/NIFI-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399121#comment-16399121 ] ASF GitHub Bot commented on NIFI-4849: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174576129 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/entity/ProcessorDiagnosticsEntity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.web.api.entity; + +import javax.xml.bind.annotation.XmlRootElement; + +import org.apache.nifi.web.api.dto.diagnostics.ProcessorDiagnosticsDTO; + +import io.swagger.annotations.ApiModelProperty; + +@XmlRootElement(name = "processorDiagnostics") --- End diff -- Yup, will do. > Add REST Endpoint for gathering Processor Diagnostics information > - > > Key: NIFI-4849 > URL: https://issues.apache.org/jira/browse/NIFI-4849 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > We need to add a REST endpoint that will use the appropriate resources to > gather the Processor Diagnostics information. Information to return should > include things like: > * Processor config > * Processor status > * Garbage Collection info > * Repo Sizes > * Connection info for connections whose source or destination is the > processor > * Controller Services that the processor is referencing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4849) Add REST Endpoint for gathering Processor Diagnostics information
[ https://issues.apache.org/jira/browse/NIFI-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399119#comment-16399119 ] ASF GitHub Bot commented on NIFI-4849: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174575890 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java --- @@ -1349,10 +1404,12 @@ private void initiateStart(final ScheduledExecutorService taskScheduler, final l LOG.debug("Successfully invoked @OnScheduled methods of {} but scheduled state is no longer STARTING so will stop processor now", processor); // can only happen if stopProcessor was called before service was transitioned to RUNNING state +activateThread(); try { ReflectionUtils.quietlyInvokeMethodsWithAnnotation(OnUnscheduled.class, processor, processContext); -} finally { --- End diff -- It shouldn't really matter -- the point of quietlyInvokeMethodsWithAnnotation is that the method will not throw any Exceptions. So whether it's in a finally block or not isn't really a big deal. > Add REST Endpoint for gathering Processor Diagnostics information > - > > Key: NIFI-4849 > URL: https://issues.apache.org/jira/browse/NIFI-4849 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > We need to add a REST endpoint that will use the appropriate resources to > gather the Processor Diagnostics information. Information to return should > include things like: > * Processor config > * Processor status > * Garbage Collection info > * Repo Sizes > * Connection info for connections whose source or destination is the > processor > * Controller Services that the processor is referencing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2468: NIFI-4849: Implemented REST Endpoint and associated...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174576129 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/entity/ProcessorDiagnosticsEntity.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.web.api.entity; + +import javax.xml.bind.annotation.XmlRootElement; + +import org.apache.nifi.web.api.dto.diagnostics.ProcessorDiagnosticsDTO; + +import io.swagger.annotations.ApiModelProperty; + +@XmlRootElement(name = "processorDiagnostics") --- End diff -- Yup, will do. ---
[jira] [Commented] (NIFI-4849) Add REST Endpoint for gathering Processor Diagnostics information
[ https://issues.apache.org/jira/browse/NIFI-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399120#comment-16399120 ] ASF GitHub Bot commented on NIFI-4849: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174576032 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/manager/StatusMerger.java --- @@ -626,6 +634,132 @@ public static void merge(final SystemDiagnosticsSnapshotDTO target, final System updatePrettyPrintedFields(target); } +public static void merge(final JVMDiagnosticsSnapshotDTO target, final JVMDiagnosticsSnapshotDTO toMerge, final long numMillis) { +if (target == null || toMerge == null) { +return; +} + +if (toMerge.getControllerDiagnostics() == null) { +target.setControllerDiagnostics(null); +} else { +merge(target.getControllerDiagnostics(), toMerge.getControllerDiagnostics()); +} + +if (toMerge.getFlowDiagnosticsDto() == null) { +target.setFlowDiagnosticsDto(null); +} else { +merge(target.getFlowDiagnosticsDto(), toMerge.getFlowDiagnosticsDto()); +} + +if (toMerge.getSystemDiagnosticsDto() == null) { +target.setSystemDiagnosticsDto(null); +} else { +merge(target.getSystemDiagnosticsDto(), toMerge.getSystemDiagnosticsDto(), numMillis); +} +} + +private static void merge(final JVMControllerDiagnosticsSnapshotDTO target, final JVMControllerDiagnosticsSnapshotDTO toMerge) { +if (toMerge == null || target == null) { +return; +} + + target.setMaxEventDrivenThreads(add(target.getMaxEventDrivenThreads(), toMerge.getMaxEventDrivenThreads())); + target.setMaxTimerDrivenThreads(add(target.getMaxTimerDrivenThreads(), toMerge.getMaxTimerDrivenThreads())); --- End diff -- I think so - can imagine it being helpful to see how many threads are available across the cluster. > Add REST Endpoint for gathering Processor Diagnostics information > - > > Key: NIFI-4849 > URL: https://issues.apache.org/jira/browse/NIFI-4849 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > We need to add a REST endpoint that will use the appropriate resources to > gather the Processor Diagnostics information. Information to return should > include things like: > * Processor config > * Processor status > * Garbage Collection info > * Repo Sizes > * Connection info for connections whose source or destination is the > processor > * Controller Services that the processor is referencing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2468: NIFI-4849: Implemented REST Endpoint and associated...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174576032 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/manager/StatusMerger.java --- @@ -626,6 +634,132 @@ public static void merge(final SystemDiagnosticsSnapshotDTO target, final System updatePrettyPrintedFields(target); } +public static void merge(final JVMDiagnosticsSnapshotDTO target, final JVMDiagnosticsSnapshotDTO toMerge, final long numMillis) { +if (target == null || toMerge == null) { +return; +} + +if (toMerge.getControllerDiagnostics() == null) { +target.setControllerDiagnostics(null); +} else { +merge(target.getControllerDiagnostics(), toMerge.getControllerDiagnostics()); +} + +if (toMerge.getFlowDiagnosticsDto() == null) { +target.setFlowDiagnosticsDto(null); +} else { +merge(target.getFlowDiagnosticsDto(), toMerge.getFlowDiagnosticsDto()); +} + +if (toMerge.getSystemDiagnosticsDto() == null) { +target.setSystemDiagnosticsDto(null); +} else { +merge(target.getSystemDiagnosticsDto(), toMerge.getSystemDiagnosticsDto(), numMillis); +} +} + +private static void merge(final JVMControllerDiagnosticsSnapshotDTO target, final JVMControllerDiagnosticsSnapshotDTO toMerge) { +if (toMerge == null || target == null) { +return; +} + + target.setMaxEventDrivenThreads(add(target.getMaxEventDrivenThreads(), toMerge.getMaxEventDrivenThreads())); + target.setMaxTimerDrivenThreads(add(target.getMaxTimerDrivenThreads(), toMerge.getMaxTimerDrivenThreads())); --- End diff -- I think so - can imagine it being helpful to see how many threads are available across the cluster. ---
[GitHub] nifi pull request #2468: NIFI-4849: Implemented REST Endpoint and associated...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2468#discussion_r174575890 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java --- @@ -1349,10 +1404,12 @@ private void initiateStart(final ScheduledExecutorService taskScheduler, final l LOG.debug("Successfully invoked @OnScheduled methods of {} but scheduled state is no longer STARTING so will stop processor now", processor); // can only happen if stopProcessor was called before service was transitioned to RUNNING state +activateThread(); try { ReflectionUtils.quietlyInvokeMethodsWithAnnotation(OnUnscheduled.class, processor, processContext); -} finally { --- End diff -- It shouldn't really matter -- the point of quietlyInvokeMethodsWithAnnotation is that the method will not throw any Exceptions. So whether it's in a finally block or not isn't really a big deal. ---
[jira] [Commented] (NIFI-4975) Add support for MongoDB GridFS
[ https://issues.apache.org/jira/browse/NIFI-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399074#comment-16399074 ] Mike Thomsen commented on NIFI-4975: [~VijayJain] I actually just submitted the PR which contains the processors for this ticket. If you would like to contribute, you can do a code review [here|https://github.com/apache/nifi/pull/2546]. > Add support for MongoDB GridFS > -- > > Key: NIFI-4975 > URL: https://issues.apache.org/jira/browse/NIFI-4975 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > [An overview |https://docs.mongodb.com/manual/core/gridfs/]of what GridFS is. > Basic CRUD processors for handling GridFS should be added to the MongoDB NAR. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4975) Add support for MongoDB GridFS
[ https://issues.apache.org/jira/browse/NIFI-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399018#comment-16399018 ] ASF GitHub Bot commented on NIFI-4975: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2546 NIFI-4975 Add GridFS processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4975 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2546.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2546 commit caef8ce496ab861e8e32224aa37fda32d43f45d2 Author: Mike Thomsen Date: 2018-03-10T17:57:28Z NIFI-4975 Add GridFS processors > Add support for MongoDB GridFS > -- > > Key: NIFI-4975 > URL: https://issues.apache.org/jira/browse/NIFI-4975 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > [An overview |https://docs.mongodb.com/manual/core/gridfs/]of what GridFS is. > Basic CRUD processors for handling GridFS should be added to the MongoDB NAR. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2546: NIFI-4975 Add GridFS processors
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2546 NIFI-4975 Add GridFS processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4975 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2546.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2546 commit caef8ce496ab861e8e32224aa37fda32d43f45d2 Author: Mike Thomsen Date: 2018-03-10T17:57:28Z NIFI-4975 Add GridFS processors ---
[jira] [Commented] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
[ https://issues.apache.org/jira/browse/NIFI-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399004#comment-16399004 ] ASF GitHub Bot commented on NIFI-4976: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2545 NIFI-4976: If unable to retrieve message content, warn an error but a… …cknowledge message. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4976 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2545.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2545 commit 6949b448316de6613c8d3281a13c1eb454193d94 Author: Mark Payne Date: 2018-03-14T18:04:03Z NIFI-4976: If unable to retrieve message content, warn an error but acknowledge message. > ConsumeJMS occasionally failing when reading TextMessage from IBM MQ > > > Key: NIFI-4976 > URL: https://issues.apache.org/jira/browse/NIFI-4976 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When reading from IBM MQ we will occasionally encounter a stack trace like > the following: > {code:java} > 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: > Failed to convert BytesMessage to byte[] > at > org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) > at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) > at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) > at > org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) > at > org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) > at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >
[GitHub] nifi pull request #2545: NIFI-4976: If unable to retrieve message content, w...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2545 NIFI-4976: If unable to retrieve message content, warn an error but a⦠â¦cknowledge message. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4976 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2545.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2545 commit 6949b448316de6613c8d3281a13c1eb454193d94 Author: Mark Payne Date: 2018-03-14T18:04:03Z NIFI-4976: If unable to retrieve message content, warn an error but acknowledge message. ---
[jira] [Commented] (MINIFICPP-414) Implement Expression Language boolean logic operations
[ https://issues.apache.org/jira/browse/MINIFICPP-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398994#comment-16398994 ] ASF GitHub Bot commented on MINIFICPP-414: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/275 @achristianson I see conflicts. they may be easy to resolve, so i can take a look if need be. > Implement Expression Language boolean logic operations > -- > > Key: MINIFICPP-414 > URL: https://issues.apache.org/jira/browse/MINIFICPP-414 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > [Boolean > Logic|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#boolean] > * > [isNull|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isnull] > * > [notNull|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#notnull] > * > [isEmpty|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isempty] > * > [equals|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#equals] > * > [equalsIgnoreCase|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#equalsignorecase] > * > [gt|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#gt] > * > [ge|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#ge] > * > [lt|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#lt] > * > [le|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#le] > * > [and|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#and] > * > [or|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#or] > * > [not|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#not] > * > [ifElse|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#ifelse] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #275: MINIFICPP-414 Added boolean Expression Language ...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/275 @achristianson I see conflicts. they may be easy to resolve, so i can take a look if need be. ---
[jira] [Commented] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
[ https://issues.apache.org/jira/browse/NIFI-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398989#comment-16398989 ] Colin Dean commented on NIFI-4974: -- Filed with jTDS: https://sourceforge.net/p/jtds/bugs/771/ I'm still concerned about the uncaught exception within NiFi. The processing of Clob/Blob/etc. is occurring within a try-with-resources but it seems like maybe there is more exception handling to be done within, especially handling the possibility that {{free()}} might throw a {{SQLFeatureNotSupportedException}} and halt processing of further columns for that result. > Uncaught AbstractMethodError when converting SQL results to Avro with JTDS > driver > - > > Key: NIFI-4974 > URL: https://issues.apache.org/jira/browse/NIFI-4974 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Windows Server 2018, MS SQL Server >Reporter: Colin Dean >Priority: Major > > I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data > from a Microsoft SQL Server installation. This is the first time I've come > upon this kind of error while using JTDS so I'm not sure if it's a defect in > JTDS or in NiFi. > {code:java} > java.lang.AbstractMethodError: null > at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) > at > org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) > at > org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) > at > org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > Looking at the JTDS source code, provided that the Github repo I'm looking at > it up to date with the SF repo, it appears that [this call to > {{free()}}|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] > is intended to generate an error from JTDS: > {code} > /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// > public void free() throws SQLException { > // TODO Auto-generated method stub > throw new AbstractMethodError(); > } > {code} > We're going to try to use the official MS SQL JDBC driver as a workaround. I > minimally want to document this in case someone else encounters this problem > or there's a way to workaround it within NiFi. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4975) Add support for MongoDB GridFS
[ https://issues.apache.org/jira/browse/NIFI-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398983#comment-16398983 ] Vijay Jain commented on NIFI-4975: -- Hi [~mike.thomsen] , I am Vijay from NIT Warangal. I have experience in java, javascript, Xml, Html, Css, Sql database, NoSql database (Cassandra) .I have interest in coding and want to contribute in open source. Please guide me through the problem and how to proceed . > Add support for MongoDB GridFS > -- > > Key: NIFI-4975 > URL: https://issues.apache.org/jira/browse/NIFI-4975 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > [An overview |https://docs.mongodb.com/manual/core/gridfs/]of what GridFS is. > Basic CRUD processors for handling GridFS should be added to the MongoDB NAR. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
[ https://issues.apache.org/jira/browse/NIFI-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398970#comment-16398970 ] Colin Dean commented on NIFI-4974: -- It looks like the problem might be in jTDS, because [{{free()}} should throw {{SQLFeatureNotSupportedException}} |https://docs.oracle.com/javase/8/docs/api/java/sql/Clob.html#free--], not AbstractMethodError when it's not implemented. > Uncaught AbstractMethodError when converting SQL results to Avro with JTDS > driver > - > > Key: NIFI-4974 > URL: https://issues.apache.org/jira/browse/NIFI-4974 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Windows Server 2018, MS SQL Server >Reporter: Colin Dean >Priority: Major > > I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data > from a Microsoft SQL Server installation. This is the first time I've come > upon this kind of error while using JTDS so I'm not sure if it's a defect in > JTDS or in NiFi. > {code:java} > java.lang.AbstractMethodError: null > at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) > at > org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) > at > org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) > at > org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > Looking at the JTDS source code, provided that the Github repo I'm looking at > it up to date with the SF repo, it appears that [this call to > {{free()}}|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] > is intended to generate an error from JTDS: > {code} > /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// > public void free() throws SQLException { > // TODO Auto-generated method stub > throw new AbstractMethodError(); > } > {code} > We're going to try to use the official MS SQL JDBC driver as a workaround. I > minimally want to document this in case someone else encounters this problem > or there's a way to workaround it within NiFi. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4976) ConsumeJMS occasionally failing when reading TextMessage from IBM MQ
Mark Payne created NIFI-4976: Summary: ConsumeJMS occasionally failing when reading TextMessage from IBM MQ Key: NIFI-4976 URL: https://issues.apache.org/jira/browse/NIFI-4976 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Assignee: Mark Payne When reading from IBM MQ we will occasionally encounter a stack trace like the following: {code:java} 2018-02-27 11:26:53,138 WARN [Timer-Driven Process Thread-186] o.a.n.c.t.ContinuallyRunProcessorTask org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[] at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42) at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:84) at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:65) at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:65) at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:119) at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:136) at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:57) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@14c194bd' at sun.reflect.GeneratedConstructorAccessor1012.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226) at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382) at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421) at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240) at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205) at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124) at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40) ... 19 common frames omitted Caused by: java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:281) at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690) at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375) ... 24 common frames omitted{code} Unfortunately, if using Client Acknowledge, we end up not acknowledging the message, since we can't read it, so it stays on the queue, and we constantly attempt to pull it again, causing the processor to essentially become 'stuck'. If such an issue arises, where we cannot retrieve the content of the message, we are better off to log an ERROR with the stack trace and then acknowledge the message even without the content. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
[ https://issues.apache.org/jira/browse/NIFI-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Dean updated NIFI-4974: - Description: I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data from a Microsoft SQL Server installation. This is the first time I've come upon this kind of error while using JTDS so I'm not sure if it's a defect in JTDS or in NiFi. {code:java} java.lang.AbstractMethodError: null at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {code} Looking at the JTDS source code, provided that the Github repo I'm looking at it up to date with the SF repo, it appears that [this call to {{free()}}|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] is intended to generate an error from JTDS: {code} /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// public void free() throws SQLException { // TODO Auto-generated method stub throw new AbstractMethodError(); } {code} We're going to try to use the official MS SQL JDBC driver as a workaround. I minimally want to document this in case someone else encounters this problem or there's a way to workaround it within NiFi. was: I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data from a Microsoft SQL Server installation. This is the first time I've come upon this kind of error while using JTDS so I'm not sure if it's a defect in JTDS or in NiFi. {code:java} java.lang.AbstractMethodError: null at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {code} Looking at the JTDS source code, provided that the Github repo I'm looking at it up to date with the SF repo, it appears that [this call to `free()`|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] is
[jira] [Commented] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
[ https://issues.apache.org/jira/browse/NIFI-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398964#comment-16398964 ] Colin Dean commented on NIFI-4974: -- Looks like the `free()` method on the Clob is being called [here|https://github.com/apache/nifi/blob/b5ca7adbb97c603cbc721e105c4fe279cdcb085b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L314]. > Uncaught AbstractMethodError when converting SQL results to Avro with JTDS > driver > - > > Key: NIFI-4974 > URL: https://issues.apache.org/jira/browse/NIFI-4974 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Windows Server 2018, MS SQL Server >Reporter: Colin Dean >Priority: Major > > I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data > from a Microsoft SQL Server installation. This is the first time I've come > upon this kind of error while using JTDS so I'm not sure if it's a defect in > JTDS or in NiFi. > {code:java} > java.lang.AbstractMethodError: null > at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) > at > org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) > at > org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) > at > org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > Looking at the JTDS source code, provided that the Github repo I'm looking at > it up to date with the SF repo, it appears that [this call to > `free()`|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] > is intended to generate an error from JTDS: > {code} > /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// > public void free() throws SQLException { > // TODO Auto-generated method stub > throw new AbstractMethodError(); > } > {code} > We're going to try to use the official MS SQL JDBC driver as a workaround. I > minimally want to document this in case someone else encounters this problem > or there's a way to workaround it within NiFi. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
[ https://issues.apache.org/jira/browse/NIFI-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398964#comment-16398964 ] Colin Dean edited comment on NIFI-4974 at 3/14/18 5:38 PM: --- Looks like the {{free()}} method on the Clob is being called [here|https://github.com/apache/nifi/blob/b5ca7adbb97c603cbc721e105c4fe279cdcb085b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L314]. was (Author: colindean): Looks like the `free()` method on the Clob is being called [here|https://github.com/apache/nifi/blob/b5ca7adbb97c603cbc721e105c4fe279cdcb085b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L314]. > Uncaught AbstractMethodError when converting SQL results to Avro with JTDS > driver > - > > Key: NIFI-4974 > URL: https://issues.apache.org/jira/browse/NIFI-4974 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Windows Server 2018, MS SQL Server >Reporter: Colin Dean >Priority: Major > > I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data > from a Microsoft SQL Server installation. This is the first time I've come > upon this kind of error while using JTDS so I'm not sure if it's a defect in > JTDS or in NiFi. > {code:java} > java.lang.AbstractMethodError: null > at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) > at > org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) > at > org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) > at > org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > Looking at the JTDS source code, provided that the Github repo I'm looking at > it up to date with the SF repo, it appears that [this call to > `free()`|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] > is intended to generate an error from JTDS: > {code} > /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// > public void free() throws SQLException { > // TODO Auto-generated method stub > throw new AbstractMethodError(); > } > {code} > We're going to try to use the official MS SQL JDBC driver as a workaround. I > minimally want to document this in case someone else encounters this problem > or there's a way to workaround it within NiFi. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4975) Add support for MongoDB GridFS
Mike Thomsen created NIFI-4975: -- Summary: Add support for MongoDB GridFS Key: NIFI-4975 URL: https://issues.apache.org/jira/browse/NIFI-4975 Project: Apache NiFi Issue Type: Improvement Reporter: Mike Thomsen Assignee: Mike Thomsen [An overview |https://docs.mongodb.com/manual/core/gridfs/]of what GridFS is. Basic CRUD processors for handling GridFS should be added to the MongoDB NAR. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4974) Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver
Colin Dean created NIFI-4974: Summary: Uncaught AbstractMethodError when converting SQL results to Avro with JTDS driver Key: NIFI-4974 URL: https://issues.apache.org/jira/browse/NIFI-4974 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.5.0 Environment: Windows Server 2018, MS SQL Server Reporter: Colin Dean I'm using the [jtds|https://jtds.sourceforge.net] driver to retrieve data from a Microsoft SQL Server installation. This is the first time I've come upon this kind of error while using JTDS so I'm not sure if it's a defect in JTDS or in NiFi. {code:java} java.lang.AbstractMethodError: null at net.sourceforge.jtds.jdbc.ClobImpl.free(ClobImpl.java:219) at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:288) at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:230) at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571) at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:218) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {code} Looking at the JTDS source code, provided that the Github repo I'm looking at it up to date with the SF repo, it appears that [this call to `free()`|https://github.com/milesibastos/jTDS/blob/master/src/main/net/sourceforge/jtds/jdbc/ClobImpl.java#L219] is intended to generate an error from JTDS: {code} /// JDBC4 demarcation, do NOT put any JDBC3 code below this line /// public void free() throws SQLException { // TODO Auto-generated method stub throw new AbstractMethodError(); } {code} We're going to try to use the official MS SQL JDBC driver as a workaround. I minimally want to document this in case someone else encounters this problem or there's a way to workaround it within NiFi. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-422) Preserve type information in Expression Language
[ https://issues.apache.org/jira/browse/MINIFICPP-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398949#comment-16398949 ] ASF GitHub Bot commented on MINIFICPP-422: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/274 > Preserve type information in Expression Language > > > Key: MINIFICPP-422 > URL: https://issues.apache.org/jira/browse/MINIFICPP-422 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > In order to improve accuracy/compatibility with Apache NiFi, type information > should be preserved in EL operations. In particular, treatment of null and > boolean values should be typed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #274: MINIFICPP-422 Refactored type system in E...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/274 ---
[jira] [Commented] (MINIFICPP-414) Implement Expression Language boolean logic operations
[ https://issues.apache.org/jira/browse/MINIFICPP-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398942#comment-16398942 ] ASF GitHub Bot commented on MINIFICPP-414: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/275 Fixed compiler errors. (Travis fail is due to a timeout on one build). > Implement Expression Language boolean logic operations > -- > > Key: MINIFICPP-414 > URL: https://issues.apache.org/jira/browse/MINIFICPP-414 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > [Boolean > Logic|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#boolean] > * > [isNull|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isnull] > * > [notNull|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#notnull] > * > [isEmpty|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#isempty] > * > [equals|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#equals] > * > [equalsIgnoreCase|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#equalsignorecase] > * > [gt|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#gt] > * > [ge|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#ge] > * > [lt|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#lt] > * > [le|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#le] > * > [and|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#and] > * > [or|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#or] > * > [not|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#not] > * > [ifElse|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#ifelse] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #275: MINIFICPP-414 Added boolean Expression Language ...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/275 Fixed compiler errors. (Travis fail is due to a timeout on one build). ---
[jira] [Commented] (NIFI-2853) Improve ListHDFS state tracking
[ https://issues.apache.org/jira/browse/NIFI-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398926#comment-16398926 ] Sivaprasanna Sethuraman commented on NIFI-2853: --- [~bende] I actually saw the code when I was going through this a few days ago but somehow when I replicate the scenario that I had mentioned above, it happens so I thought it has something to do with the isConfigurationRestored() check and also I dint quite understand what isConfigurationRestored() does or its logic. Can you please try to replicate the above scenario on your end to confirm? > Improve ListHDFS state tracking > --- > > Key: NIFI-2853 > URL: https://issues.apache.org/jira/browse/NIFI-2853 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Bryan Bende >Priority: Minor > > Currently ListHDFS tracks two properties in state management, > "listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the > directory property now supports expression language which means the directory > being listed could dynamically change on any execution of the processor. > The processor should be changed to store state specific to the directory that > was listed, for example "listing.timestamp.dir1" and "emitted.timestamp.dir1". > This would also help in a clustered scenario... currently ListHDFS has to be > run on primary node only, otherwise each node will be overwriting each others > state and producing unexpected results. With the above improvement, if the > directory evaluated to a unique path for each node, it would store the state > of each of those path separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-543) Provide extensions a way to indicate that they can run only on primary node, if clustered
[ https://issues.apache.org/jira/browse/NIFI-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398917#comment-16398917 ] Bryan Bende commented on NIFI-543: -- [~sivaprasanna] I don't have the cycles right now to review this, but regarding the comment about 'Versioned Processor'... Is there really anything we need to add there? The versioned processor already has the scheduling strategy that the processor was set to, this annotation on the NiFi side just forces it so that the strategy would only ever be primary node. I'm not sure if there is a need for versioned processor to also know that the processor itself was annotated with this. > Provide extensions a way to indicate that they can run only on primary node, > if clustered > - > > Key: NIFI-543 > URL: https://issues.apache.org/jira/browse/NIFI-543 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework, Documentation & Website, Extensions >Reporter: Mark Payne >Assignee: Sivaprasanna Sethuraman >Priority: Major > > There are Processors that are known to be problematic if run from multiple > nodes simultaneously. These processors should be able to use a > @PrimaryNodeOnly annotation (or something similar) to indicate that they can > be scheduled to run only on primary node if run in a cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-2853) Improve ListHDFS state tracking
[ https://issues.apache.org/jira/browse/NIFI-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398906#comment-16398906 ] Bryan Bende commented on NIFI-2853: --- [~sivaprasanna] I'm not sure this scenario is accurate... Any time the directory or filter is changed in the processor, the state tracking resets: [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/ListHDFS.java#L204-L206] So in your example when you changed from "/tmp/sub-dir" to "/tmp" it would reset and pick up everything again. I don't think there is anything to change about the state tracking. We can either close the JIRA, or apply the default scheduling strategy that Pierre suggested. > Improve ListHDFS state tracking > --- > > Key: NIFI-2853 > URL: https://issues.apache.org/jira/browse/NIFI-2853 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Bryan Bende >Priority: Minor > > Currently ListHDFS tracks two properties in state management, > "listing.timestamp" and "emitted.timestamp". In the 1.0.0 release, the > directory property now supports expression language which means the directory > being listed could dynamically change on any execution of the processor. > The processor should be changed to store state specific to the directory that > was listed, for example "listing.timestamp.dir1" and "emitted.timestamp.dir1". > This would also help in a clustered scenario... currently ListHDFS has to be > run on primary node only, otherwise each node will be overwriting each others > state and producing unexpected results. With the above improvement, if the > directory evaluated to a unique path for each node, it would store the state > of each of those path separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-422) Preserve type information in Expression Language
[ https://issues.apache.org/jira/browse/MINIFICPP-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398838#comment-16398838 ] ASF GitHub Bot commented on MINIFICPP-422: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/274 Compile errors fixed. > Preserve type information in Expression Language > > > Key: MINIFICPP-422 > URL: https://issues.apache.org/jira/browse/MINIFICPP-422 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > In order to improve accuracy/compatibility with Apache NiFi, type information > should be preserved in EL operations. In particular, treatment of null and > boolean values should be typed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #274: MINIFICPP-422 Refactored type system in EL to pr...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/274 Compile errors fixed. ---
[jira] [Commented] (NIFI-4959) HandleHttpRequest processor doesn't close/release incomplete message error
[ https://issues.apache.org/jira/browse/NIFI-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398756#comment-16398756 ] ASF GitHub Bot commented on NIFI-4959: -- GitHub user mgaido91 opened a pull request: https://github.com/apache/nifi/pull/2544 NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? Tested manually - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? NA - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? NA - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? NA - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? NA ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mgaido91/nifi NIFI-4959 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2544.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2544 commit 15f0a04a1b2b4600ccf2b43b83f0ec157dbfd500 Author: Marco Gaido Date: 2018-03-14T15:27:17Z NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException > HandleHttpRequest processor doesn't close/release incomplete message error > -- > > Key: NIFI-4959 > URL: https://issues.apache.org/jira/browse/NIFI-4959 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0, 1.6.0 > Environment: Linux, all versions of nifi-1.X >Reporter: Wynner >Priority: Major > > I am doing some testing with the HandleHttpRequest processor. My specific > test, involves sending an incomplete request and closing the connection from > the sending system. Initially, it throws the error I expect, but it keeps > throwing the error over and over based on the request expiration configured > in the StandardHttpContextMap controller service. > The only way to stop the error message is to stop the processor. In my test, > I saw one failed request throw an error six times before I stopped the > processor. > It doesn't seems to terminate the request on the NiFi side. > Sample HTTP request > > POST/ HTTP/ 1.1 > Host: foo.com > Content-Type: text/plain > Content-Length: 130 > say=Hi&to-Mom > > I use the telnet command to connect to the system with the processor > listening, post the message above , close the connection, and then the > processor starts throws the following error indefinitely > 2018-03-10 01:36:37,111 ERROR [Timer-Driven Process Thread-6] > o.a.n.p.standard.HandleHttpRequest > HandleHttpRequest[id=0d8547f7-0162-1000-9b84-129af2382259] > HandleHttpRequest[id=0d8547f7-0162-1000-9b84-129af2382259] failed to process > session due to org.apache.nifi.processor.exception.FlowFileAccessException: > Failed to import data from > HttpInputOverHTTP@46e7d12e[c=15,q=0,[0]=null,s=EARLY_EOF] for > StandardFlowFileRecord[uuid=32bb182d-f619-4b98-b6f8-c1ed50c2736a,claim=,offset=0,name=9714775822613527,size=0] > due to org.apache.nifi.processor.exception.FlowFileAccessException: Unable > to create ContentClaim due to org.eclipse.jetty.io.EofException: Early EOF: {} >
[GitHub] nifi pull request #2544: NIFI-4959: Remove flowfiles and close connection fo...
GitHub user mgaido91 opened a pull request: https://github.com/apache/nifi/pull/2544 NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? Tested manually - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? NA - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? NA - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? NA - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? NA ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mgaido91/nifi NIFI-4959 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2544.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2544 commit 15f0a04a1b2b4600ccf2b43b83f0ec157dbfd500 Author: Marco Gaido Date: 2018-03-14T15:27:17Z NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException ---
[jira] [Commented] (MINIFICPP-422) Preserve type information in Expression Language
[ https://issues.apache.org/jira/browse/MINIFICPP-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398681#comment-16398681 ] ASF GitHub Bot commented on MINIFICPP-422: -- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/274 Taking a look. > Preserve type information in Expression Language > > > Key: MINIFICPP-422 > URL: https://issues.apache.org/jira/browse/MINIFICPP-422 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > In order to improve accuracy/compatibility with Apache NiFi, type information > should be preserved in EL operations. In particular, treatment of null and > boolean values should be typed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4743) Suppress Nulls for PutElasticsearchHttpRecord
[ https://issues.apache.org/jira/browse/NIFI-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398703#comment-16398703 ] ASF GitHub Bot commented on NIFI-4743: -- Github user robertrbruno commented on the issue: https://github.com/apache/nifi/pull/2501 @MikeThomsen changes look good to me. > Suppress Nulls for PutElasticsearchHttpRecord > - > > Key: NIFI-4743 > URL: https://issues.apache.org/jira/browse/NIFI-4743 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Bruno >Assignee: Mike Thomsen >Priority: Minor > Attachments: NullSuppression.java, PutElasticsearchHttpRecord.java > > > Would be useful for PutElasticsearchHttpRecord to allow you to suppress NULL > values in the JSON that is inserted into ES much like the JsonRecordSetWriter > allows you to do. Perhaps PutElasticsearchHttpRecord could some how make use > of JsonRecordSetWriter so it would inherit this functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4743) Suppress Nulls for PutElasticsearchHttpRecord
[ https://issues.apache.org/jira/browse/NIFI-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398693#comment-16398693 ] ASF GitHub Bot commented on NIFI-4743: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2501 @pvillard31 @robertrbruno I checked in a new commit based on the feedback. Please take a look when you have a min. > Suppress Nulls for PutElasticsearchHttpRecord > - > > Key: NIFI-4743 > URL: https://issues.apache.org/jira/browse/NIFI-4743 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Bruno >Assignee: Mike Thomsen >Priority: Minor > Attachments: NullSuppression.java, PutElasticsearchHttpRecord.java > > > Would be useful for PutElasticsearchHttpRecord to allow you to suppress NULL > values in the JSON that is inserted into ES much like the JsonRecordSetWriter > allows you to do. Perhaps PutElasticsearchHttpRecord could some how make use > of JsonRecordSetWriter so it would inherit this functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4743) Suppress Nulls for PutElasticsearchHttpRecord
[ https://issues.apache.org/jira/browse/NIFI-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398689#comment-16398689 ] ASF GitHub Bot commented on NIFI-4743: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2501#discussion_r174483646 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java --- @@ -284,6 +306,17 @@ public void onTrigger(final ProcessContext context, final ProcessSession session return; } +final NullSuppression suppression; --- End diff -- Done. > Suppress Nulls for PutElasticsearchHttpRecord > - > > Key: NIFI-4743 > URL: https://issues.apache.org/jira/browse/NIFI-4743 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Bruno >Assignee: Mike Thomsen >Priority: Minor > Attachments: NullSuppression.java, PutElasticsearchHttpRecord.java > > > Would be useful for PutElasticsearchHttpRecord to allow you to suppress NULL > values in the JSON that is inserted into ES much like the JsonRecordSetWriter > allows you to do. Perhaps PutElasticsearchHttpRecord could some how make use > of JsonRecordSetWriter so it would inherit this functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4743) Suppress Nulls for PutElasticsearchHttpRecord
[ https://issues.apache.org/jira/browse/NIFI-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398690#comment-16398690 ] ASF GitHub Bot commented on NIFI-4743: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2501#discussion_r174483710 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/NullSuppression.java --- @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.elasticsearch; + +public enum NullSuppression { +ALWAYS_SUPPRESS, +NEVER_SUPPRESS, +SUPPRESS_MISSING --- End diff -- Done. > Suppress Nulls for PutElasticsearchHttpRecord > - > > Key: NIFI-4743 > URL: https://issues.apache.org/jira/browse/NIFI-4743 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Bruno >Assignee: Mike Thomsen >Priority: Minor > Attachments: NullSuppression.java, PutElasticsearchHttpRecord.java > > > Would be useful for PutElasticsearchHttpRecord to allow you to suppress NULL > values in the JSON that is inserted into ES much like the JsonRecordSetWriter > allows you to do. Perhaps PutElasticsearchHttpRecord could some how make use > of JsonRecordSetWriter so it would inherit this functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-144) NPE when accessing for metadata of 'latest' version and no versions exists
[ https://issues.apache.org/jira/browse/NIFIREG-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398695#comment-16398695 ] ASF GitHub Bot commented on NIFIREG-144: GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/105 NIFIREG-144 Fixing NPE when retrieving latest snapshot when no versio… …ns exist You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-144 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/105.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #105 commit 0cb2752c8164913cf97d01c705a5faab54eb0288 Author: Bryan Bende Date: 2018-03-14T14:44:27Z NIFIREG-144 Fixing NPE when retrieving latest snapshot when no versions exist > NPE when accessing for metadata of 'latest' version and no versions exists > -- > > Key: NIFIREG-144 > URL: https://issues.apache.org/jira/browse/NIFIREG-144 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: Bryan Bende >Priority: Minor > > {code:java} > 2018-02-14 10:59:35,879 ERROR [NiFi Registry Web Server-82] > o.a.n.r.web.mapper.ThrowableMapper An unexpected error has occurred: > java.lang.NullPointerException. Returning Internal Server Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.registry.service.DataModelMapper.map(DataModelMapper.java:106) > ~[nifi-registry-framework-0.2.0-SNAPSHOT.jar:0.2.0-SNAPSHOT] > at > org.apache.nifi.registry.service.RegistryService.getLatestFlowSnapshotMetadata(RegistryService.java:742) > ~[nifi-registry-framework-0.2.0-SNAPSHOT.jar:0.2.0-SNAPSHOT] > at > org.apache.nifi.registry.service.RegistryService$$FastClassBySpringCGLIB$$274de08.invoke() > ~[nifi-registry-framework-0.2.0-SNAPSHOT.jar:0.2.0-SNAPSHOT] > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > ~[spring-core-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747) > ~[spring-aop-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > ~[spring-aop-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) > ~[spring-tx-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) > ~[spring-tx-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) > ~[spring-aop-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) > ~[spring-aop-5.0.2.RELEASE.jar:5.0.2.RELEASE] > at > org.apache.nifi.registry.service.RegistryService$$EnhancerBySpringCGLIB$$6e647495.getLatestFlowSnapshotMetadata() > ~[nifi-registry-framework-0.2.0-SNAPSHOT.jar:0.2.0-SNAPSHOT] > at > org.apache.nifi.registry.web.api.BucketFlowResource.getLatestFlowVersionMetadata(BucketFlowResource.java:368) > ~[classes/:na] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[na:1.8.0_144] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[na:1.8.0_144] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[na:1.8.0_144] > at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144] > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) > ~[jersey-server-2.26.jar:na] > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) > ~[jersey-server-2.26.jar:na] > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) > ~[jersey-server-2.26.jar:na] > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) > ~[jersey-server-2.26.jar:na] > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMet
[GitHub] nifi issue #2501: NIFI-4743 Added configurable null suppression to PutElasti...
Github user robertrbruno commented on the issue: https://github.com/apache/nifi/pull/2501 @MikeThomsen changes look good to me. ---
[GitHub] nifi-registry pull request #105: NIFIREG-144 Fixing NPE when retrieving late...
GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/105 NIFIREG-144 Fixing NPE when retrieving latest snapshot when no versio⦠â¦ns exist You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-144 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/105.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #105 commit 0cb2752c8164913cf97d01c705a5faab54eb0288 Author: Bryan Bende Date: 2018-03-14T14:44:27Z NIFIREG-144 Fixing NPE when retrieving latest snapshot when no versions exist ---
[GitHub] nifi issue #2501: NIFI-4743 Added configurable null suppression to PutElasti...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2501 @pvillard31 @robertrbruno I checked in a new commit based on the feedback. Please take a look when you have a min. ---
[GitHub] nifi pull request #2501: NIFI-4743 Added configurable null suppression to Pu...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2501#discussion_r174483646 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java --- @@ -284,6 +306,17 @@ public void onTrigger(final ProcessContext context, final ProcessSession session return; } +final NullSuppression suppression; --- End diff -- Done. ---
[GitHub] nifi pull request #2501: NIFI-4743 Added configurable null suppression to Pu...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2501#discussion_r174483710 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/NullSuppression.java --- @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.elasticsearch; + +public enum NullSuppression { +ALWAYS_SUPPRESS, +NEVER_SUPPRESS, +SUPPRESS_MISSING --- End diff -- Done. ---
[GitHub] nifi-minifi-cpp issue #274: MINIFICPP-422 Refactored type system in EL to pr...
Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/274 Taking a look. ---
[jira] [Created] (NIFI-4973) RPG port merging
Matt Gilman created NIFI-4973: - Summary: RPG port merging Key: NIFI-4973 URL: https://issues.apache.org/jira/browse/NIFI-4973 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Matt Gilman When clustered, the available ports that should be returned to the client should only include ports that all nodes have discovered. It appears that currently, it's returning the union and not the intersection. The code in question is in RemoteProcessGroupEntityMerger#mergeDtos(...) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-426) Update bootstrap to download libcurl-openssl on centos and rhel
marco polo created MINIFICPP-426: Summary: Update bootstrap to download libcurl-openssl on centos and rhel Key: MINIFICPP-426 URL: https://issues.apache.org/jira/browse/MINIFICPP-426 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: marco polo Assignee: marco polo -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-397) Implement RouteOnAttribute
[ https://issues.apache.org/jira/browse/MINIFICPP-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398626#comment-16398626 ] ASF GitHub Bot commented on MINIFICPP-397: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/268 > Implement RouteOnAttribute > -- > > Key: MINIFICPP-397 > URL: https://issues.apache.org/jira/browse/MINIFICPP-397 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > RouteOnAttribute is notably missing from MiNiFi - C++ and should be > implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #268: MINIFICPP-397 Added implementation of Rou...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/268 ---
[jira] [Commented] (NIFI-4912) Update jackson version to latest stable
[ https://issues.apache.org/jira/browse/NIFI-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398625#comment-16398625 ] ASF GitHub Bot commented on NIFI-4912: -- Github user jomach commented on the issue: https://github.com/apache/nifi/pull/2494 +1 > Update jackson version to latest stable > --- > > Key: NIFI-4912 > URL: https://issues.apache.org/jira/browse/NIFI-4912 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Derek Straka >Assignee: Derek Straka >Priority: Major > Fix For: 1.6.0 > > > The current jackson version is out of date and contains several CVEs as well > as outstanding bugs. Update to the latest stable version which at the time > of writing is 2.9.4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2494: NIFI-4912: Update jackson version to latest stable version...
Github user jomach commented on the issue: https://github.com/apache/nifi/pull/2494 +1 ---
[jira] [Commented] (MINIFICPP-397) Implement RouteOnAttribute
[ https://issues.apache.org/jira/browse/MINIFICPP-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398618#comment-16398618 ] ASF GitHub Bot commented on MINIFICPP-397: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/268 Will build and merge now > Implement RouteOnAttribute > -- > > Key: MINIFICPP-397 > URL: https://issues.apache.org/jira/browse/MINIFICPP-397 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > RouteOnAttribute is notably missing from MiNiFi - C++ and should be > implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #268: MINIFICPP-397 Added implementation of RouteOnAtt...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/268 Will build and merge now ---
[jira] [Commented] (MINIFICPP-333) Implement string search functions in Expression Language
[ https://issues.apache.org/jira/browse/MINIFICPP-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398615#comment-16398615 ] ASF GitHub Bot commented on MINIFICPP-333: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/273 > Implement string search functions in Expression Language > > > Key: MINIFICPP-333 > URL: https://issues.apache.org/jira/browse/MINIFICPP-333 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > Implement these: > startsWith > endsWith > contains > in > find > matches > indexOf > lastIndexOf -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #273: MINIFICPP-333 Added string searching func...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/273 ---
[jira] [Resolved] (MINIFICPP-424) Update CAInfo manually so user does not have to update the cert bundle on their machine
[ https://issues.apache.org/jira/browse/MINIFICPP-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri resolved MINIFICPP-424. --- Resolution: Fixed Fix Version/s: 0.5.0 > Update CAInfo manually so user does not have to update the cert bundle on > their machine > --- > > Key: MINIFICPP-424 > URL: https://issues.apache.org/jira/browse/MINIFICPP-424 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > > Update CAInfo manually so user does not have to update the cert bundle on > their machine. We should do this for the user's convenience. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #279: Minificpp 424
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/279 ---
[GitHub] nifi-minifi-cpp pull request #279: Minificpp 424
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/279#discussion_r174459253 --- Diff: extensions/http-curl/client/HTTPClient.cpp --- @@ -334,6 +334,8 @@ void HTTPClient::configure_secure_connection(CURL *http_session) { logger_->log_debug("Using certificate file %s", ssl_context_service_->getCertificateFile()); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_FUNCTION, &configure_ssl_context); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_DATA, static_cast(ssl_context_service_.get())); + curl_easy_setopt(http_session, CURLOPT_CAINFO, 0); --- End diff -- Okay, thanks for clarifying. Looks good to me. Will merge. ---
[GitHub] nifi-minifi-cpp pull request #279: Minificpp 424
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/279#discussion_r174445579 --- Diff: extensions/http-curl/client/HTTPClient.cpp --- @@ -334,6 +334,8 @@ void HTTPClient::configure_secure_connection(CURL *http_session) { logger_->log_debug("Using certificate file %s", ssl_context_service_->getCertificateFile()); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_FUNCTION, &configure_ssl_context); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_DATA, static_cast(ssl_context_service_.get())); + curl_easy_setopt(http_session, CURLOPT_CAINFO, 0); --- End diff -- There is little documentation about this but this is to ensure we don't have any ambiguous error messages. Users should be using libcurl-openssl; however, RHEL variants don't have this built into the repos, so we will have to download the source and build it. As a result, the default implementation uses NSS which is not how we've implemented the code. ---
[GitHub] nifi-minifi-cpp pull request #279: Minificpp 424
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/279#discussion_r174443984 --- Diff: extensions/http-curl/client/HTTPClient.cpp --- @@ -334,6 +334,8 @@ void HTTPClient::configure_secure_connection(CURL *http_session) { logger_->log_debug("Using certificate file %s", ssl_context_service_->getCertificateFile()); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_FUNCTION, &configure_ssl_context); curl_easy_setopt(http_session, CURLOPT_SSL_CTX_DATA, static_cast(ssl_context_service_.get())); + curl_easy_setopt(http_session, CURLOPT_CAINFO, 0); --- End diff -- I was able to find in the curl docs (https://curl.haxx.se/libcurl/c/CURLOPT_CAINFO.html) for what this behavior does for NSS and curl 7.55+ and get how this applies but didn't see any such docs for openssl variants or other versions. Is the behavior pretty consistent from that standpoint? ---
[jira] [Commented] (MINIFICPP-402) Add docs for UpdateAttribute
[ https://issues.apache.org/jira/browse/MINIFICPP-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398487#comment-16398487 ] ASF GitHub Bot commented on MINIFICPP-402: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/278 > Add docs for UpdateAttribute > > > Key: MINIFICPP-402 > URL: https://issues.apache.org/jira/browse/MINIFICPP-402 > Project: NiFi MiNiFi C++ > Issue Type: Documentation >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > Docs need to be added to README/PROCESSORS.md for UpdateAttribute. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #278: MINIFICPP-402 Add docs for UpdateAttribut...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/278 ---