[GitHub] nifi-registry issue #134: NIFIREG-192: Implement REGISTRY_START event
Github user jdye64 commented on the issue: https://github.com/apache/nifi-registry/pull/134 Sure will. I will update that right now and make another commit ---
[GitHub] nifi-registry issue #133: NIFIREG-190
Github user jdye64 commented on the issue: https://github.com/apache/nifi-registry/pull/133 I think if the EventType is null, which I'm like you I don't know if it ever could be?? Then the whitelisting logic should be ignored and the Event Handlers should all be invoked. That makes the most sense to me. ---
[GitHub] nifi-registry issue #133: NIFIREG-190
Github user jdye64 commented on the issue: https://github.com/apache/nifi-registry/pull/133 Ok. After reading through all of these I'm thinking the best way to handle this and have it operate the way I had envisioned is to take the approach of adding the handleEvent() method in the EventProvider interface as outlined above. I'm going to start on that now and just remove the Abstract class all together. ---
[GitHub] nifi-registry issue #133: NIFIREG-190
Github user jdye64 commented on the issue: https://github.com/apache/nifi-registry/pull/133 I had contemplated placing the logic that checked if the hook provider should be invoked for an event or not right above https://github.com/apache/nifi-registry/blob/7e4e9713406f3bb3ac1544b6ad69fbdd437f794c/nifi-registry-framework/src/main/java/org/apache/nifi/registry/event/EventService.java#L73 but decided to place that logic in each hook provider in case someone wanted to write their own custom hook provider and want to apply different white listing logic that would allow them to circumvent such filtering. How do you both feel about that? ---
[GitHub] nifi-registry issue #133: NIFIREG-190
Github user jdye64 commented on the issue: https://github.com/apache/nifi-registry/pull/133 Looks like the Travis CI build is failing trying to extract the npm binary for building the web ui section. This doesn't have anything to do with the PR so don't believe the build is truly failing here... ---
[GitHub] nifi-registry pull request #134: NIFIREG-192: Implement REGISTRY_START event
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-registry/pull/134 NIFIREG-192: Implement REGISTRY_START event You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-registry NIFIREG-192 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/134.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #134 commit 06c89db48d77ebaf408128c893c97f4c9fccb17c Author: Jeremy Dyer Date: 2018-08-09T02:37:12Z NIFIREG-192: Implement REGISTRY_START event ---
[GitHub] nifi-registry pull request #133: NIFIREG-190
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-registry/pull/133 NIFIREG-190 You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-registry NIFIREG-190 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #133 commit 6821dc10bd633fec324c2ca82616269dbc8cda0b Author: Jeremy Dyer Date: 2018-08-08T17:41:42Z NIFIREG-190 Add support for whitelist filtering Event Hook providers commit 673734cf0a2298a68c6d7921fe8b626a6706567f Author: Jeremy Dyer Date: 2018-08-08T17:43:06Z Fixed git issue where I had accidentally removed tracking for NifiRegistryApiApplication.java commit a527fbaf1ca31a9e03aa4093b9ca639057343cc0 Author: Jeremy Dyer Date: 2018-08-08T17:44:11Z Removed unused imports ---
[GitHub] nifi issue #2932: NIFI-5480: Use FlowController's maps of components in orde...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2932 @mcgilman I had actually already started reviewing this and am almost done. If its ok with you I'll review it ---
[GitHub] nifi-minifi-cpp issue #371: MINIFICPP-556 Added initial snapcraft build
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/371 @phrocker were you able to resolve your issues? I didn't experience any problems during my testing. ---
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r175308013
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that
contains the access token. IETF OAuth2 spec default is 'access_token' if your
API provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API
provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME =
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that
contains the expire in. IETF OAuth2 spec default is 'expire_
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r175308002
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
--- End diff --
That makes sense Andy. All properties now support expression language
---
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r175098169
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/httpclient/OAuthHTTPConnectionClient.java
---
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth.httpclient;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.HttpURLConnection;
+import java.net.URL;
+import java.net.URLConnection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.oltu.oauth2.client.HttpClient;
+import org.apache.oltu.oauth2.client.request.OAuthClientRequest;
+import org.apache.oltu.oauth2.client.response.OAuthAccessTokenResponse;
+import org.apache.oltu.oauth2.client.response.OAuthClientResponse;
+import org.apache.oltu.oauth2.common.OAuth;
+import org.apache.oltu.oauth2.common.exception.OAuthProblemException;
+import org.apache.oltu.oauth2.common.exception.OAuthSystemException;
+import org.apache.oltu.oauth2.common.token.BasicOAuthToken;
+import org.apache.oltu.oauth2.common.token.OAuthToken;
+import org.apache.oltu.oauth2.common.utils.OAuthUtils;
+import org.json.JSONObject;
+
+public class OAuthHTTPConnectionClient
+implements HttpClient {
+
+private String accessTokenName = null;
+private String tokenTypeName = null;
+private String scopeName = null;
+private String expireInName = null;
+private String expireTimeName = null;
+
+public OAuthHTTPConnectionClient(String accessTokenName, String
tokenTypeName, String scopeName, String expireInName, String expireTimeName) {
+this.accessTokenName = accessTokenName;
+this.tokenTypeName = tokenTypeName;
+this.scopeName = scopeName;
+this.expireInName = expireInName;
+this.expireTimeName = expireTimeName;
+}
+
+@Override
+public T execute(OAuthClientRequest
request, Map headers,
+String requestMethod, Class responseClass) throws
OAuthSystemException, OAuthProblemException {
+
+InputStream responseBody = null;
+URLConnection c;
+Map> responseHeaders = new HashMap>();
+int responseCode;
+try {
+URL url = new URL(request.getLocationUri());
+
+c = url.openConnection();
+responseCode = -1;
+if (c instanceof HttpURLConnection) {
+HttpURLConnection httpURLConnection = (HttpURLConnection)
c;
+
+if (headers != null && !headers.isEmpty()) {
+for (Map.Entry header :
headers.entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (request.getHeaders() != null) {
+for (Map.Entry header :
request.getHeaders().entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (OAuthUtils.isEmpty(requestMethod)) {
+
httpURLConnection.setRequestMethod(OAuth.HttpMethod.GET);
+} else {
+httpURLConnection.setRequestMethod(requestMethod);
+setRequestBody(request, requestMethod,
httpURLConnection);
+}
+
+httpURLConnection.connect();
+
+InputStream inputStream;
+responseCode = httpURLConnection.getResponseCode();
+if (responseCode == 400 || responseCode == 405 ||
response
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r174933983
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/httpclient/OAuthHTTPConnectionClient.java
---
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth.httpclient;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.HttpURLConnection;
+import java.net.URL;
+import java.net.URLConnection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.oltu.oauth2.client.HttpClient;
+import org.apache.oltu.oauth2.client.request.OAuthClientRequest;
+import org.apache.oltu.oauth2.client.response.OAuthAccessTokenResponse;
+import org.apache.oltu.oauth2.client.response.OAuthClientResponse;
+import org.apache.oltu.oauth2.common.OAuth;
+import org.apache.oltu.oauth2.common.exception.OAuthProblemException;
+import org.apache.oltu.oauth2.common.exception.OAuthSystemException;
+import org.apache.oltu.oauth2.common.token.BasicOAuthToken;
+import org.apache.oltu.oauth2.common.token.OAuthToken;
+import org.apache.oltu.oauth2.common.utils.OAuthUtils;
+import org.json.JSONObject;
+
+public class OAuthHTTPConnectionClient
+implements HttpClient {
+
+private String accessTokenName = null;
+private String tokenTypeName = null;
+private String scopeName = null;
+private String expireInName = null;
+private String expireTimeName = null;
+
+public OAuthHTTPConnectionClient(String accessTokenName, String
tokenTypeName, String scopeName, String expireInName, String expireTimeName) {
+this.accessTokenName = accessTokenName;
+this.tokenTypeName = tokenTypeName;
+this.scopeName = scopeName;
+this.expireInName = expireInName;
+this.expireTimeName = expireTimeName;
+}
+
+@Override
+public T execute(OAuthClientRequest
request, Map headers,
+String requestMethod, Class responseClass) throws
OAuthSystemException, OAuthProblemException {
+
+InputStream responseBody = null;
+URLConnection c;
+Map> responseHeaders = new HashMap>();
+int responseCode;
+try {
+URL url = new URL(request.getLocationUri());
+
+c = url.openConnection();
+responseCode = -1;
+if (c instanceof HttpURLConnection) {
+HttpURLConnection httpURLConnection = (HttpURLConnection)
c;
+
+if (headers != null && !headers.isEmpty()) {
+for (Map.Entry header :
headers.entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (request.getHeaders() != null) {
+for (Map.Entry header :
request.getHeaders().entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (OAuthUtils.isEmpty(requestMethod)) {
+
httpURLConnection.setRequestMethod(OAuth.HttpMethod.GET);
+} else {
+httpURLConnection.setRequestMethod(requestMethod);
+setRequestBody(request, requestMethod,
httpURLConnection);
+}
+
+httpURLConnection.connect();
+
+InputStream inputStream;
+responseCode = httpURLConnection.getResponseCode();
+if (responseCode == 400 || responseCode == 405 ||
response
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r174933433
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/httpclient/OAuthHTTPConnectionClient.java
---
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth.httpclient;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.net.HttpURLConnection;
+import java.net.URL;
+import java.net.URLConnection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.oltu.oauth2.client.HttpClient;
+import org.apache.oltu.oauth2.client.request.OAuthClientRequest;
+import org.apache.oltu.oauth2.client.response.OAuthAccessTokenResponse;
+import org.apache.oltu.oauth2.client.response.OAuthClientResponse;
+import org.apache.oltu.oauth2.common.OAuth;
+import org.apache.oltu.oauth2.common.exception.OAuthProblemException;
+import org.apache.oltu.oauth2.common.exception.OAuthSystemException;
+import org.apache.oltu.oauth2.common.token.BasicOAuthToken;
+import org.apache.oltu.oauth2.common.token.OAuthToken;
+import org.apache.oltu.oauth2.common.utils.OAuthUtils;
+import org.json.JSONObject;
+
+public class OAuthHTTPConnectionClient
+implements HttpClient {
+
+private String accessTokenName = null;
+private String tokenTypeName = null;
+private String scopeName = null;
+private String expireInName = null;
+private String expireTimeName = null;
+
+public OAuthHTTPConnectionClient(String accessTokenName, String
tokenTypeName, String scopeName, String expireInName, String expireTimeName) {
+this.accessTokenName = accessTokenName;
+this.tokenTypeName = tokenTypeName;
+this.scopeName = scopeName;
+this.expireInName = expireInName;
+this.expireTimeName = expireTimeName;
+}
+
+@Override
+public T execute(OAuthClientRequest
request, Map headers,
+String requestMethod, Class responseClass) throws
OAuthSystemException, OAuthProblemException {
+
+InputStream responseBody = null;
+URLConnection c;
+Map> responseHeaders = new HashMap>();
+int responseCode;
+try {
+URL url = new URL(request.getLocationUri());
+
+c = url.openConnection();
+responseCode = -1;
+if (c instanceof HttpURLConnection) {
+HttpURLConnection httpURLConnection = (HttpURLConnection)
c;
+
+if (headers != null && !headers.isEmpty()) {
+for (Map.Entry header :
headers.entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (request.getHeaders() != null) {
+for (Map.Entry header :
request.getHeaders().entrySet()) {
+
httpURLConnection.addRequestProperty(header.getKey(), header.getValue());
+}
+}
+
+if (OAuthUtils.isEmpty(requestMethod)) {
+
httpURLConnection.setRequestMethod(OAuth.HttpMethod.GET);
+} else {
+httpURLConnection.setRequestMethod(requestMethod);
+setRequestBody(request, requestMethod,
httpURLConnection);
+}
+
+httpURLConnection.connect();
+
+InputStream inputStream;
+responseCode = httpURLConnection.getResponseCode();
+if (responseCode == 400 || responseCode == 405 ||
response
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r174931088
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/OAuth2ClientCredentialsGrantControllerService.java
---
@@ -0,0 +1,167 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ * Created on 7/25/17.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.oauth.httpclient.OAuthHTTPConnectionClient;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.oltu.oauth2.client.HttpClient;
+import org.apache.oltu.oauth2.client.request.OAuthClientRequest;
+import org.apache.oltu.oauth2.common.exception.OAuthProblemException;
+import org.apache.oltu.oauth2.common.exception.OAuthSystemException;
+import org.apache.oltu.oauth2.common.message.types.GrantType;
+
+@Tags({ "oauth2", "client", "secret", "post"})
+@CapabilityDescription("POSTs the ClientId and ClientSecret to the OAuth2
authentication server to retrieve the" +
+" access token. The access token is stored locally in the
controller service and used by processors " +
+"referencing this controller service.")
+public class OAuth2ClientCredentialsGrantControllerService
+extends AbstractOAuthControllerService
+implements OAuth2ClientService {
+
+public static final PropertyDescriptor CLIENT_ID = new
PropertyDescriptor
+.Builder().name("OAuth2 Client ID")
+.displayName("OAuth2 Client ID")
+.description("OAuth2 Client ID passed to the authorization
server")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CLIENT_SECRET = new
PropertyDescriptor
+.Builder().name("OAuth2 Client Secret")
+.displayName("OAuth2 Client Secret")
+.description("OAuth2 Client Secret that will be passed to the
authorization server in exchange for an access token")
+.sensitive(true)
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+private static final List properties;
+
+static {
+final List props = new ArrayList<>();
+props.add(AUTH_SERVER_URL);
+props.add(CLIENT_ID);
+props.add(CLIENT_SECRET);
+props.add(RESPONSE_ACCESS_TOKEN_FIELD_NAME);
+props.add(RESPONSE_EXPIRE_TIME_FIELD_NAME);
+props.add(RESPONSE_EXPIRE_IN_FIELD_NAME);
+props.add(RESPONSE_SCOPE_FIELD_NAME);
+props.add(RESPONSE_TOKEN_TYPE_FIELD_NAME);
+properties = Collections.unmodifiableList(props);
+}
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+return properties;
+}
+
+private String clientId = null;
+private String clientSecret = null;
+private Base64.Encoder enc = Base64.getEncoder();
+
+/**
+ * @param context the configuration context
+ * @throws InitializationException if unable to create a database
connection
+ */
+
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r172295045
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
--- End diff --
Good point I will change the name and enable expression language by default
---
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r172294695
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that
contains the access token. IETF OAuth2 spec default is 'access_token' if your
API provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API
provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME =
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that
contains the expire in. IETF OAuth2 spec default is 'expire_
[GitHub] nifi-minifi-cpp pull request #37: MINIFI-171 Dynamic Properties support for ...
Github user jdye64 closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/37 ---
[GitHub] nifi issue #2439: NIFI-4824 - Allow user to specific host ports on Docker Im...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 Alright!! Congratulations! Thank you for the effort and keep up the good work! ---
[GitHub] nifi issue #2085: NIFI-4246 - Client Credentials Grant based OAuth2 Controll...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2085 @alopresto Is there anything I can do to help move this along? ---
[GitHub] nifi issue #2439: NIFI-4824 - Allow user to specific host ports on Docker Im...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2439 +1 everything built and ran well for me. Merging now. ---
[GitHub] nifi pull request #2439: NIFI-4824 - Allow user to specific host ports on Do...
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2439#discussion_r165158617 --- Diff: nifi-docker/dockerhub/README.md --- @@ -46,12 +46,22 @@ The configuration scripts are suitable for at least 1.4.0+. The minimum to run a NiFi instance is as follows: docker run --name nifi \ - -p 18080:8080 \ + -p 8080:8080 \ -d \ apache/nifi:latest -This will provide a running instance, exposing the instance UI to the host system on at port 18080, -viewable at `http://localhost:18080/nifi`. +This will provide a running instance, exposing the instance UI to the host system on at port 8080, +viewable at `http://localhost:8080/nifi`. + +You can also pass in environment variables to change the NiFi communication ports and hostname using the Docker '-e' switch as follows: + +docker run --name nifi \ + -p 9090:9090 \ + -d \ + -e NIFI_WEB_HTTP_PORT='9090' + apache/nifi:latest + +For a list of the environment variables recognised in this build, look into the .sh/secure.sh and .sh/start.sh scripts --- End diff -- Enjoy the English touch with "recognised" instead of "recognized" =) ---
[GitHub] nifi issue #1016: NIFI-2724 New JMX Processor
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1016 @brianburnett once you make the recommended changes I can help you with the rebasing effort. ---
[GitHub] nifi issue #2441: NIFI-4823 Made pretty printing configurable in GetMongo.
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2441 +1 looks good to me. Will merge now ---
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r160197767
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that
contains the access token. IETF OAuth2 spec default is 'access_token' if your
API provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API
provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME =
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that
contains the expire in. IETF OAuth2 spec default is 'expire_
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2085#discussion_r160197726
--- Diff:
nifi-nar-bundles/nifi-oauth-bundle/nifi-oauth/src/main/java/org/apache/nifi/oauth/AbstractOAuthControllerService.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.oauth;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+
+
+public abstract class AbstractOAuthControllerService
+extends AbstractControllerService implements OAuth2ClientService {
+
+protected String accessToken = null;
+protected String refreshToken = null;
+protected String tokenType = null;
+protected long expiresIn = -1;
+protected long expiresTime = -1;
+protected long lastResponseTimestamp = -1;
+protected Map extraHeaders = new HashMap();
+protected String authUrl = null;
+protected long expireTimeSafetyNetSeconds = -1;
+protected String accessTokenRespName = null;
+protected String expireTimeRespName = null;
+protected String expireInRespName = null;
+protected String tokenTypeRespName = null;
+protected String scopeRespName = null;
+
+public static final PropertyDescriptor AUTH_SERVER_URL = new
PropertyDescriptor
+.Builder().name("OAuth2 Authorization Server URL")
+.displayName("OAuth2 Authorization Server")
+.description("OAuth2 Authorization Server that grants access
to the protected resources on the behalf of the resource owner.")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor
RESPONSE_ACCESS_TOKEN_FIELD_NAME = new PropertyDescriptor
+.Builder().name("JSON response 'access_token' name")
+.displayName("JSON response 'access_token' name")
+.description("Name of the field in the JSON response that
contains the access token. IETF OAuth2 spec default is 'access_token' if your
API provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("access_token")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_TIME_FIELD_NAME
= new PropertyDescriptor
+.Builder().name("JSON response 'expire_time' name")
+.displayName("JSON response 'expire_time' name")
+.description("Name of the field in the JSON response that
contains the expire time. IETF OAuth2 spec default is 'expire_time' if your API
provider's" +
+" response field is different this is where you can
change that.")
+.defaultValue("expire_time")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor RESPONSE_EXPIRE_IN_FIELD_NAME =
new PropertyDescriptor
+.Builder().name("JSON response 'expire_in' name")
+.displayName("JSON response 'expire_in' name")
+.description("Name of the field in the JSON response that
contains the expire in. IETF OAuth2 spec default is 'expire_
[GitHub] nifi-minifi-cpp pull request #52: MINIFI-218: Support for GPSD integration
Github user jdye64 closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/52 ---
[GitHub] nifi-minifi-cpp issue #52: MINIFI-218: Support for GPSD integration
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/52 Sorry for the late response here folks. I'm going to close this issue because as stated it has already been taken care of elsewhere. ---
[GitHub] nifi issue #2245: NIFI-4496: Added JacksonCSVRecordReader to allow choice of...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2245 @mattyb149 I'm seeing invalid output when I run run an existing flow with this PR. I had an existing flow that used ConvertRecord and Apache Commons CSV. That was working fine and giving me the output I expected. However when I switched to using the Jackson implementation all of the output was empty. I have attached a screenshot from my debugger session in hopes that will help shed some light into what is going on. https://user-images.githubusercontent.com/2127235/32498256-32f8ffc6-c39d-11e7-86dd-cde8f7d3a758.png";> ---
[GitHub] nifi-minifi-cpp issue #52: MINIFI-218: Support for GPSD integration
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/52 This poor PR has been neglected. I think that now @phrocker PR for extensions has been committed its a good time to make another pass at this and place it in that extensions format ---
[GitHub] nifi-minifi-cpp issue #142: MINIFI-372: Replace leveldb with RocksDB
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/142 @phrocker this was a big one. Sorry it took so long to work but with the changes wanted to test several things. The PR looks good to me in its current state and will commit in a few minutes. ---
[GitHub] nifi-minifi-cpp issue #143: MINIFICPP-254: Resolve odd naming of isRunning v...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/143 @fstakem thanks for pointing this out to us. Just reviewed and all looks good. Will commit in a few minutes ---
[GitHub] nifi issue #2085: NIFI-4246 - Client Credentials Grant based OAuth2 Controll...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2085 @joewitt rebased and incremented bundle version from 1.4.0-SNAPSHOT to 1.5.0-SNAPSHOT ---
[GitHub] nifi issue #2085: NIFI-4246 - Client Credentials Grant based OAuth2 Controll...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/2085 @alopresto I tested it against tubemogl [1] and freewheel [2]. I obviously can't share those sensitive credentials here but would something like http://term.ie/oauth/example/ suffice? [1] - https://www.tubemogul.com/ [2] - http://freewheel.tv/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2085: NIFI-4246 - Client Credentials Grant based OAuth2 C...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi/pull/2085 NIFI-4246 - Client Credentials Grant based OAuth2 Controller Service Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi NIFI-4246 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2085.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2085 commit 6a63e544168f2db0059bab9b155ef9e3bd034531 Author: Jeremy Dyer Date: 2017-07-25T14:07:24Z NIFI-4246 - Client Credentials Grant based OAuth2 Controller Service --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #116: Minifi 341 - Tailfile Delimiter for input
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/116 Minifi 341 - Tailfile Delimiter for input Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-341 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/116.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #116 commit 15f571980ccce197202d9292de429f203ff6b67a Author: Jeremy Dyer Date: 2017-06-23T02:29:21Z MINIFI-341. Introduce delimiter to TailFile to delimit incoming data appropriately. commit 37741406d82e969ab8eeaabd5df196ad3de6eb5a Author: Jeremy Dyer Date: 2017-06-23T13:09:00Z Input delimiter changed from std::string to char commit b8bdd639ad261a16a083a7ce830de7f425e83bcc Author: Jeremy Dyer Date: 2017-06-28T19:24:33Z Updates to TailFile delimiter logic commit 2491a70fa2538859995c5ef82578c7a3ec112f94 Author: Jeremy Dyer Date: 2017-06-29T15:24:01Z Updates for TailFile delimiter --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #88: MINIFI-286: Fix compilation error when GCC versio...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/88 I like just removing the conditional expression. I'm going to update to that now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #88: MINIFI-286: Fix compilcation error when GC...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/88 MINIFI-286: Fix compilcation error when GCC version is greater than 4.9 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-286 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/88.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #88 commit 7c6c85179d5a4acf60bd7a2b95b6d9173da62d29 Author: jdye64 Date: 2017-05-02T22:25:42Z MINIFI-286: Fix compilcation error when GCC version is greater than 4.9 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1597: NIFI-3609: ConnectWebSocket auto session recovery
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1597#discussion_r114116475
--- Diff:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketClient.java
---
@@ -135,27 +177,81 @@ public void stopClient() throws Exception {
@Override
public void connect(final String clientId) throws IOException {
+connect(clientId, null);
+}
+
+private void connect(final String clientId, String sessionId) throws
IOException {
+
+connectionLock.lock();
-final WebSocketMessageRouter router;
try {
-router = routers.getRouterOrFail(clientId);
-} catch (WebSocketConfigurationException e) {
-throw new IllegalStateException("Failed to get router due to:
" + e, e);
+final WebSocketMessageRouter router;
+try {
+router = routers.getRouterOrFail(clientId);
+} catch (WebSocketConfigurationException e) {
+throw new IllegalStateException("Failed to get router due
to: " + e, e);
+}
+final RoutingWebSocketListener listener = new
RoutingWebSocketListener(router);
+listener.setSessionId(sessionId);
+
+final ClientUpgradeRequest request = new
ClientUpgradeRequest();
+final Future connect = client.connect(listener,
webSocketUri, request);
+getLogger().info("Connecting to : {}", new
Object[]{webSocketUri});
+
+final Session session;
+try {
+session = connect.get(connectionTimeoutMillis,
TimeUnit.MILLISECONDS);
+} catch (Exception e) {
+throw new IOException("Failed to connect " + webSocketUri
+ " due to: " + e, e);
+}
+getLogger().info("Connected, session={}", new
Object[]{session});
+activeSessions.put(clientId, listener.getSessionId());
+
+} finally {
+connectionLock.unlock();
}
-final RoutingWebSocketListener listener = new
RoutingWebSocketListener(router);
-final ClientUpgradeRequest request = new ClientUpgradeRequest();
-final Future connect = client.connect(listener,
webSocketUri, request);
-getLogger().info("Connecting to : {}", new Object[]{webSocketUri});
+}
+
+private Map activeSessions = new ConcurrentHashMap<>();
-final Session session;
+void maintainSessions() throws Exception {
+if (client == null) {
+return;
+}
+
+connectionLock.lock();
+
+final ComponentLog logger = getLogger();
try {
-session = connect.get(connectionTimeoutMillis,
TimeUnit.MILLISECONDS);
-} catch (Exception e) {
-throw new IOException("Failed to connect " + webSocketUri + "
due to: " + e, e);
+// Loop through existing sessions and reconnect.
+for (String clientId : activeSessions.keySet()) {
+final WebSocketMessageRouter router;
+try {
+router = routers.getRouterOrFail(clientId);
+} catch (final WebSocketConfigurationException e) {
+if (logger.isDebugEnabled()) {
+logger.debug("The clientId {} is no longer active.
Discarding the clientId.", new Object[]{clientId});
+}
+activeSessions.remove(clientId);
+continue;
+}
+
+final String sessionId = activeSessions.get(clientId);
+// If this session is stil alive, do nothing.
+if (!router.containsSession(sessionId)) {
+// This session is no longer active, reconnect it.
+// If it fails, the sessionId will remain in
activeSessions, and retries later.
--- End diff --
More a question but should we limit the number of times this is attempted
before the sessionId is just removed? I could see an argument for either way
but curious your thought here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1597: NIFI-3609: ConnectWebSocket auto session recovery
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1597#discussion_r114116075
--- Diff:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketClient.java
---
@@ -81,6 +87,16 @@
.defaultValue("3 sec")
.build();
+public static final PropertyDescriptor SESSION_MAINTENANCE_INTERVAL =
new PropertyDescriptor.Builder()
+.name("session-maintenance-interval")
+.displayName("Session Maintenance Interval")
+.description("The interval between session maintenance
activities.")
--- End diff --
Could we add some more details around what this is doing? I get its the
interval for the maintenance thread but I'm worried that end users might not
really understand what that is? Maybe a brief sentence about what this
"maintenance" interval is actually doing and why it is necessary so they
understand how to use it a little better.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1597: NIFI-3609: ConnectWebSocket auto session recovery
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1597#discussion_r114116276
--- Diff:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketClient.java
---
@@ -81,6 +87,16 @@
.defaultValue("3 sec")
.build();
+public static final PropertyDescriptor SESSION_MAINTENANCE_INTERVAL =
new PropertyDescriptor.Builder()
+.name("session-maintenance-interval")
+.displayName("Session Maintenance Interval")
+.description("The interval between session maintenance
activities.")
+.required(true)
+.expressionLanguageSupported(true)
+.addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+.defaultValue("10 sec")
--- End diff --
Since this is invoking maintainSessions() and that is effectively
reconnecting sessions would it make sense to make this default value less than
the default value of CONNECTION_TIMEOUT just so by default this would prevent
connections from timing out at all? This would help in the case that the client
becomes inactive for longer than the default CONNECTION_TIMEOUT and help ensure
messages are not missed being delivered to the client.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1597: NIFI-3609: ConnectWebSocket auto session recovery
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1597 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1595 My reasoning for adding the extra relationship was for end users to understand that certain directories or files had improper permissions and have a visible way to see that with the extra relationship. However your right we could just add something like a "error.code" and "error.message" attributes and route to failure. I would like the "error.code" for sure however since that would be easy to route against while error.message might contain different messages from the exceptions when dependency versions change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1595 @trixpan refactoring due to NIFI-3204 #1561 which caused some conflicts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1595 @trixpan I tried talking it out but that didn't seem to solve the conflict?? Will try more forceful procedures when I get off the road and near a computer ;) thanks for taking a look at this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1649: Support for websocket multiplexing to all existing ...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi/pull/1649 Support for websocket multiplexing to all existing websocket connections (Think chat to all clients instead of individual person). The core change was a change in WebSocketMessageRouter.java where if a sessionId is not present the message is sent to all connected clients. So the key is leaving the sessionId to empty or null to send to all clients. If the sessionId is specified the message will be sent just to that session specified. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi NIFI-3318 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1649.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1649 commit d444d0addd454b619e917465aeab9b1cf8cc7ac4 Author: Jeremy Dyer Date: 2017-04-04T18:46:17Z Support for websocket multiplexing to all existing websocket connections (Think chat to all clients instead of individual person). The core change was a change in WebSocketMessageRouter.java where if a sessionId is not present the message is sent to all connected clients. So the key is leaving the sessionId to empty or null to send to all clients. If the sessionId is specified the message will be sent just to that session specified. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #929: NIFI-2613 - Apache POI processor to convert Excel documents...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/929 @jvwing In regards to error cases here are a list of things which could make this fail. - Input data is incorrect. AKA not .xlsx file - Empty document. I guess this isn't an error but I think you get what I'm saying there just wouldn't be anything to do. - User could specify a list of "desired-sheets" that are not present in the document. Once again not really failure but of course no success relationships would be triggered on the original relationship. - The processor will currently only process strings (this includes numerical values, just the POI naming convention) and will ignore things like graphs, images, binary data, etc. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1595: NIFI-3600 Improve logging and relationship routing ...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi/pull/1595 NIFI-3600 Improve logging and relationship routing for failures in De⦠â¦leteHDFS Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi NIFI-3600 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1595.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105259757
--- Diff: libminifi/include/FlowControlProtocol.h ---
@@ -45,297 +51,269 @@ class FlowController;
//! FlowControl Protocol Msg Type
typedef enum {
- REGISTER_REQ, // Device Register Request from device to server which
contain device serial number, current running flow YAML version
- REGISTER_RESP, // Device Register Respond from server to device, may
contain new flow.YAML from server ask device to apply and also device report
interval
- REPORT_REQ, // Period Device Report from device to server which contain
device serial number, current running flow YAML name/version and other period
report info
- REPORT_RESP, // Report Respond from server to device, may ask device to
update flow YAML or processor property
- MAX_FLOW_CONTROL_MSG_TYPE
+ REGISTER_REQ, // Device Register Request from device to server which
contain device serial number, current running flow YAML version
+ REGISTER_RESP, // Device Register Respond from server to device, may
contain new flow.YAML from server ask device to apply and also device report
interval
+ REPORT_REQ, // Period Device Report from device to server which contain
device serial number, current running flow YAML name/version and other period
report info
+ REPORT_RESP, // Report Respond from server to device, may ask device to
update flow YAML or processor property
+ MAX_FLOW_CONTROL_MSG_TYPE
} FlowControlMsgType;
//! FlowControl Protocol Msg Type String
-static const char *FlowControlMsgTypeStr[MAX_FLOW_CONTROL_MSG_TYPE] =
-{
- "REGISTER_REQ",
- "REGISTER_RESP",
- "REPORT_REQ",
- "REPORT_RESP"
-};
+static const char *FlowControlMsgTypeStr[MAX_FLOW_CONTROL_MSG_TYPE] = {
+"REGISTER_REQ", "REGISTER_RESP", "REPORT_REQ", "REPORT_RESP" };
//! Flow Control Msg Type to String
-inline const char *FlowControlMsgTypeToStr(FlowControlMsgType type)
-{
- if (type < MAX_FLOW_CONTROL_MSG_TYPE)
- return FlowControlMsgTypeStr[type];
- else
- return NULL;
+inline const char *FlowControlMsgTypeToStr(FlowControlMsgType type) {
+ if (type < MAX_FLOW_CONTROL_MSG_TYPE)
+return FlowControlMsgTypeStr[type];
+ else
+return NULL;
}
//! FlowControll Protocol Msg ID (Some Messages are fix length, Some are
variable length (TLV)
typedef enum {
- //Fix length 8 bytes: client to server in register request, required
field
- FLOW_SERIAL_NUMBER,
- // Flow YAML name TLV: client to server in register request and report
request, required field
- FLOW_YML_NAME,
- // Flow YAML content, TLV: server to client in register respond, option
field in case server want to ask client to load YAML from server
- FLOW_YML_CONTENT,
- // Fix length, 4 bytes Report interval in msec: server to client in
register respond, option field
- REPORT_INTERVAL,
- // Processor Name TLV: server to client in report respond, option
field in case server want to ask client to update processor property
- PROCESSOR_NAME,
- // Processor Property Name TLV: server to client in report respond,
option field in case server want to ask client to update processor property
- PROPERTY_NAME,
- // Processor Property Value TLV: server to client in report respond,
option field in case server want to ask client to update processor property
- PROPERTY_VALUE,
- // Report Blob TLV: client to server in report request, option field in
case client want to pickyback the report blob in report request to server
- REPORT_BLOB,
- MAX_FLOW_MSG_ID
+ //Fix length 8 bytes: client to server in register request, required
field
+ FLOW_SERIAL_NUMBER,
+ // Flow YAML name TLV: client to server in register request and report
request, required field
+ FLOW_YML_NAME,
+ // Flow YAML content, TLV: server to client in register respond, option
field in case server want to ask client to load YAML from server
+ FLOW_YML_CONTENT,
+ // Fix length, 4 bytes Report interval in msec: server to client in
register respond, option field
+ REPORT_INTERVAL,
+ // Processor Name TLV: server to client in report respond, option field
in case server want to ask client to update processor property
+ PROCESSOR_NAME,
+ // Processor Property Name TLV: server to client in report respond,
option field in case server want to ask client to update processor property
+ PROPERTY_NAME,
+ // Processor Property Value TLV: server to client in report respond,
option field in case server want to ask client to update processor
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105259321
--- Diff: libminifi/include/Connection.h ---
@@ -28,174 +28,168 @@
#include
#include
-#include "FlowFileRecord.h"
-#include "Logger.h"
-#include "Relationship.h"
+#include "core/Connectable.h"
+#include "core/Record.h"
+#include "core/logging/Logger.h"
+#include "core/Relationship.h"
+#include "core/Connectable.h"
-//! Forwarder declaration
-class Processor;
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
//! Connection Class
-class Connection
-{
-public:
- //! Constructor
- /*!
-* Create a new processor
-*/
- explicit Connection(std::string name, uuid_t uuid = NULL, uuid_t
srcUUID = NULL, uuid_t destUUID = NULL);
- //! Destructor
- virtual ~Connection() {}
- //! Set Connection Name
- void setName(std::string name) {
- _name = name;
- }
- //! Get Process Name
- std::string getName(void) {
- return (_name);
- }
- //! Set UUID
- void setUUID(uuid_t uuid) {
- uuid_copy(_uuid, uuid);
- }
- //! Set Source Processor UUID
- void setSourceProcessorUUID(uuid_t uuid) {
- uuid_copy(_srcUUID, uuid);
- }
- //! Set Destination Processor UUID
- void setDestinationProcessorUUID(uuid_t uuid) {
- uuid_copy(_destUUID, uuid);
- }
- //! Get Source Processor UUID
- void getSourceProcessorUUID(uuid_t uuid) {
- uuid_copy(uuid, _srcUUID);
- }
- //! Get Destination Processor UUID
- void getDestinationProcessorUUID(uuid_t uuid) {
- uuid_copy(uuid, _destUUID);
- }
- //! Get UUID
- bool getUUID(uuid_t uuid) {
- if (uuid)
- {
- uuid_copy(uuid, _uuid);
- return true;
- }
- else
- return false;
- }
- //! Set Connection Source Processor
- void setSourceProcessor(Processor *source) {
- _srcProcessor = source;
- }
- // ! Get Connection Source Processor
- Processor *getSourceProcessor() {
- return _srcProcessor;
- }
- //! Set Connection Destination Processor
- void setDestinationProcessor(Processor *dest) {
- _destProcessor = dest;
- }
- // ! Get Connection Destination Processor
- Processor *getDestinationProcessor() {
- return _destProcessor;
- }
- //! Set Connection relationship
- void setRelationship(Relationship relationship) {
- _relationship = relationship;
- }
- // ! Get Connection relationship
- Relationship getRelationship() {
- return _relationship;
- }
- //! Set Max Queue Size
- void setMaxQueueSize(uint64_t size)
- {
- _maxQueueSize = size;
- }
- //! Get Max Queue Size
- uint64_t getMaxQueueSize()
- {
- return _maxQueueSize;
- }
- //! Set Max Queue Data Size
- void setMaxQueueDataSize(uint64_t size)
- {
- _maxQueueDataSize = size;
- }
- //! Get Max Queue Data Size
- uint64_t getMaxQueueDataSize()
- {
- return _maxQueueDataSize;
- }
- //! Set Flow expiration duration in millisecond
- void setFlowExpirationDuration(uint64_t duration)
- {
- _expiredDuration = duration;
- }
- //! Get Flow expiration duration in millisecond
- uint64_t getFlowExpirationDuration()
- {
- return _expiredDuration;
- }
- //! Check whether the queue is empty
- bool isEmpty();
- //! Check whether the queue is full to apply back pressure
- bool isFull();
- //! Get queue size
- uint64_t getQueueSize() {
- std::lock_guard lock(_mtx);
- return _queue.size();
- }
- //! Get queue data size
- uint64_t getQueueDataSize()
- {
- return _maxQueueDataSize;
- }
- //! Put the flow file into queue
- void put(FlowFileRecord *flow);
- //! Poll the flow file from queue, the expired flow file record also
being returned
- FlowFileRecord *poll(std::set &expiredFlowRecords);
- //! Drain the flow records
- void drain();
+class Connection : public core::Connectable,
+public std::enable_shared_from_this {
+ public:
+ //! Constructor
+ /*!
+ * Create a new processor
+ */
+ explicit Connection(std::string name, uuid_t uuid = NU
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105254417 --- Diff: libminifi/include/Connection.h --- @@ -28,174 +28,168 @@ #include #include -#include "FlowFileRecord.h" -#include "Logger.h" -#include "Relationship.h" +#include "core/Connectable.h" +#include "core/Record.h" --- End diff -- What was the reasoning for changing the naming from "FlowFileRecord" to "Record"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105261172
--- Diff: libminifi/include/SchedulingAgent.h ---
@@ -28,72 +28,90 @@
#include
#include
#include "utils/TimeUtil.h"
-#include "Logger.h"
+#include "core/logging/Logger.h"
#include "Configure.h"
#include "FlowFileRecord.h"
-#include "Logger.h"
-#include "Processor.h"
-#include "ProcessContext.h"
+#include "core/logging/Logger.h"
+#include "core/Processor.h"
+#include "core/ProcessContext.h"
+
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+
//! SchedulingAgent Class
-class SchedulingAgent
-{
-public:
- //! Constructor
- /*!
-* Create a new processor
-*/
- SchedulingAgent() {
- configure_ = Configure::getConfigure();
- logger_ = Logger::getLogger();
- _running = false;
- }
- //! Destructor
- virtual ~SchedulingAgent()
- {
+class SchedulingAgent {
+ public:
+ //! Constructor
+ /*!
+ * Create a new processor
+ */
+ SchedulingAgent(std::shared_ptr repo) {
+configure_ = Configure::getConfigure();
+logger_ = logging::Logger::getLogger();
+_running = false;
--- End diff --
Can you move "_" to end of class data member to adhere to Google C++ Style
guide? Same for all variables here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105261283
--- Diff: libminifi/include/SchedulingAgent.h ---
@@ -28,72 +28,90 @@
#include
#include
#include "utils/TimeUtil.h"
-#include "Logger.h"
+#include "core/logging/Logger.h"
#include "Configure.h"
#include "FlowFileRecord.h"
-#include "Logger.h"
-#include "Processor.h"
-#include "ProcessContext.h"
+#include "core/logging/Logger.h"
+#include "core/Processor.h"
+#include "core/ProcessContext.h"
+
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+
//! SchedulingAgent Class
-class SchedulingAgent
-{
-public:
- //! Constructor
- /*!
-* Create a new processor
-*/
- SchedulingAgent() {
- configure_ = Configure::getConfigure();
- logger_ = Logger::getLogger();
- _running = false;
- }
- //! Destructor
- virtual ~SchedulingAgent()
- {
+class SchedulingAgent {
+ public:
+ //! Constructor
+ /*!
+ * Create a new processor
+ */
+ SchedulingAgent(std::shared_ptr repo) {
+configure_ = Configure::getConfigure();
+logger_ = logging::Logger::getLogger();
+_running = false;
+repo_ = repo;
+ }
+ //! Destructor
+ virtual ~SchedulingAgent() {
+
+ }
+ //! onTrigger, return whether the yield is need
+ bool onTrigger(
+ std::shared_ptr processor,
+ core::ProcessContext *processContext,
+ core::ProcessSessionFactory *sessionFactory);
+ //! Whether agent has work to do
+ bool hasWorkToDo(std::shared_ptr processor);
+ //! Whether the outgoing need to be backpressure
+ bool hasTooMuchOutGoing(
+ std::shared_ptr processor);
+ //! start
+ void start() {
+_running = true;
+ }
+ //! stop
+ void stop() {
+_running = false;
+ }
- }
- //! onTrigger, return whether the yield is need
- bool onTrigger(Processor *processor, ProcessContext *processContext,
ProcessSessionFactory *sessionFactory);
- //! Whether agent has work to do
- bool hasWorkToDo(Processor *processor);
- //! Whether the outgoing need to be backpressure
- bool hasTooMuchOutGoing(Processor *processor);
- //! start
- void start() {
- _running = true;
- }
- //! stop
- void stop() {
- _running = false;
- }
+ public:
+ //! schedule, overwritten by different DrivenSchedulingAgent
+ virtual void schedule(
+ std::shared_ptr processor) = 0;
+ //! unschedule, overwritten by different DrivenSchedulingAgent
+ virtual void unschedule(
+ std::shared_ptr processor) = 0;
-public:
- //! schedule, overwritten by different DrivenSchedulingAgent
- virtual void schedule(Processor *processor) = 0;
- //! unschedule, overwritten by different DrivenSchedulingAgent
- virtual void unschedule(Processor *processor) = 0;
+ SchedulingAgent(const SchedulingAgent &parent) = delete;
+ SchedulingAgent &operator=(const SchedulingAgent &parent) = delete;
+ protected:
+ //! Logger
+ std::shared_ptr logger_;
+ //! Configure
+ Configure *configure_;
+ //! Mutex for protection
+ std::mutex _mtx;
+ //! Whether it is running
+ std::atomic _running;
+ //! AdministrativeYieldDuration
+ int64_t _administrativeYieldDuration;
+ //! BoredYieldDuration
+ int64_t _boredYieldDuration;
--- End diff --
Can you move "_" to end of class data member to adhere to Google C++ Style
guide? Same for all variables here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105260453
--- Diff: libminifi/include/FlowFileRecord.h ---
@@ -32,198 +32,138 @@
#include
#include "utils/TimeUtil.h"
-#include "Logger.h"
+#include "core/logging/Logger.h"
#include "ResourceClaim.h"
+#include "Connection.h"
+#include "core/Record.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
-class ProcessSession;
-class Connection;
#define DEFAULT_FLOWFILE_PATH "."
//! FlowFile Attribute
-enum FlowAttribute
-{
- //! The flowfile's path indicates the relative directory to which a
FlowFile belongs and does not contain the filename
- PATH = 0,
- //! The flowfile's absolute path indicates the absolute directory to
which a FlowFile belongs and does not contain the filename
- ABSOLUTE_PATH,
- //! The filename of the FlowFile. The filename should not contain any
directory structure.
- FILENAME,
- //! A unique UUID assigned to this FlowFile.
- UUID,
- //! A numeric value indicating the FlowFile priority
- priority,
- //! The MIME Type of this FlowFile
- MIME_TYPE,
- //! Specifies the reason that a FlowFile is being discarded
- DISCARD_REASON,
- //! Indicates an identifier other than the FlowFile's UUID that is
known to refer to this FlowFile.
- ALTERNATE_IDENTIFIER,
- MAX_FLOW_ATTRIBUTES
+enum FlowAttribute {
+ //! The flowfile's path indicates the relative directory to which a
FlowFile belongs and does not contain the filename
+ PATH = 0,
+ //! The flowfile's absolute path indicates the absolute directory to
which a FlowFile belongs and does not contain the filename
+ ABSOLUTE_PATH,
+ //! The filename of the FlowFile. The filename should not contain any
directory structure.
+ FILENAME,
+ //! A unique UUID assigned to this FlowFile.
+ UUID,
+ //! A numeric value indicating the FlowFile priority
+ priority,
+ //! The MIME Type of this FlowFile
+ MIME_TYPE,
+ //! Specifies the reason that a FlowFile is being discarded
+ DISCARD_REASON,
+ //! Indicates an identifier other than the FlowFile's UUID that is known
to refer to this FlowFile.
+ ALTERNATE_IDENTIFIER,
+ MAX_FLOW_ATTRIBUTES
};
//! FlowFile Attribute Key
-static const char *FlowAttributeKeyArray[MAX_FLOW_ATTRIBUTES] =
-{
- "path",
- "absolute.path",
- "filename",
- "uuid",
- "priority",
- "mime.type",
- "discard.reason",
- "alternate.identifier"
-};
+static const char *FlowAttributeKeyArray[MAX_FLOW_ATTRIBUTES] = { "path",
+"absolute.path", "filename", "uuid", "priority", "mime.type",
+"discard.reason", "alternate.identifier" };
//! FlowFile Attribute Enum to Key
-inline const char *FlowAttributeKey(FlowAttribute attribute)
-{
- if (attribute < MAX_FLOW_ATTRIBUTES)
- return FlowAttributeKeyArray[attribute];
- else
- return NULL;
+inline const char *FlowAttributeKey(FlowAttribute attribute) {
+ if (attribute < MAX_FLOW_ATTRIBUTES)
+return FlowAttributeKeyArray[attribute];
+ else
+return NULL;
}
//! FlowFile IO Callback functions for input and output
//! throw exception for error
-class InputStreamCallback
-{
-public:
- virtual void process(std::ifstream *stream) = 0;
+class InputStreamCallback {
+ public:
+ virtual void process(std::ifstream *stream) = 0;
};
-class OutputStreamCallback
-{
-public:
- virtual void process(std::ofstream *stream) = 0;
+class OutputStreamCallback {
+ public:
+ virtual void process(std::ofstream *stream) = 0;
};
-
//! FlowFile Record Class
-class FlowFileRecord
-{
- friend class ProcessSession;
-public:
- //! Constructor
- /*!
-* Create a new flow record
-*/
- FlowFileRecord(std::map attributes,
ResourceClaim *claim = NULL);
- //! Destructor
- virtual ~FlowFileRecord();
- //! addAttribute key is enum
- bool addAttribute(FlowAttribute key, std::string value);
- //! addAttribute key is string
- bool addAttribute(std::string key, std::string value);
- //! removeAttribute key is enum
- bool removeAttribute(FlowAttribute key);
- //! removeAttribute ke
[GitHub] nifi-minifi-cpp pull request #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/63#discussion_r105259428
--- Diff: libminifi/include/Connection.h ---
@@ -28,174 +28,168 @@
#include
#include
-#include "FlowFileRecord.h"
-#include "Logger.h"
-#include "Relationship.h"
+#include "core/Connectable.h"
+#include "core/Record.h"
+#include "core/logging/Logger.h"
+#include "core/Relationship.h"
+#include "core/Connectable.h"
-//! Forwarder declaration
-class Processor;
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
//! Connection Class
-class Connection
-{
-public:
- //! Constructor
- /*!
-* Create a new processor
-*/
- explicit Connection(std::string name, uuid_t uuid = NULL, uuid_t
srcUUID = NULL, uuid_t destUUID = NULL);
- //! Destructor
- virtual ~Connection() {}
- //! Set Connection Name
- void setName(std::string name) {
- _name = name;
- }
- //! Get Process Name
- std::string getName(void) {
- return (_name);
- }
- //! Set UUID
- void setUUID(uuid_t uuid) {
- uuid_copy(_uuid, uuid);
- }
- //! Set Source Processor UUID
- void setSourceProcessorUUID(uuid_t uuid) {
- uuid_copy(_srcUUID, uuid);
- }
- //! Set Destination Processor UUID
- void setDestinationProcessorUUID(uuid_t uuid) {
- uuid_copy(_destUUID, uuid);
- }
- //! Get Source Processor UUID
- void getSourceProcessorUUID(uuid_t uuid) {
- uuid_copy(uuid, _srcUUID);
- }
- //! Get Destination Processor UUID
- void getDestinationProcessorUUID(uuid_t uuid) {
- uuid_copy(uuid, _destUUID);
- }
- //! Get UUID
- bool getUUID(uuid_t uuid) {
- if (uuid)
- {
- uuid_copy(uuid, _uuid);
- return true;
- }
- else
- return false;
- }
- //! Set Connection Source Processor
- void setSourceProcessor(Processor *source) {
- _srcProcessor = source;
- }
- // ! Get Connection Source Processor
- Processor *getSourceProcessor() {
- return _srcProcessor;
- }
- //! Set Connection Destination Processor
- void setDestinationProcessor(Processor *dest) {
- _destProcessor = dest;
- }
- // ! Get Connection Destination Processor
- Processor *getDestinationProcessor() {
- return _destProcessor;
- }
- //! Set Connection relationship
- void setRelationship(Relationship relationship) {
- _relationship = relationship;
- }
- // ! Get Connection relationship
- Relationship getRelationship() {
- return _relationship;
- }
- //! Set Max Queue Size
- void setMaxQueueSize(uint64_t size)
- {
- _maxQueueSize = size;
- }
- //! Get Max Queue Size
- uint64_t getMaxQueueSize()
- {
- return _maxQueueSize;
- }
- //! Set Max Queue Data Size
- void setMaxQueueDataSize(uint64_t size)
- {
- _maxQueueDataSize = size;
- }
- //! Get Max Queue Data Size
- uint64_t getMaxQueueDataSize()
- {
- return _maxQueueDataSize;
- }
- //! Set Flow expiration duration in millisecond
- void setFlowExpirationDuration(uint64_t duration)
- {
- _expiredDuration = duration;
- }
- //! Get Flow expiration duration in millisecond
- uint64_t getFlowExpirationDuration()
- {
- return _expiredDuration;
- }
- //! Check whether the queue is empty
- bool isEmpty();
- //! Check whether the queue is full to apply back pressure
- bool isFull();
- //! Get queue size
- uint64_t getQueueSize() {
- std::lock_guard lock(_mtx);
- return _queue.size();
- }
- //! Get queue data size
- uint64_t getQueueDataSize()
- {
- return _maxQueueDataSize;
- }
- //! Put the flow file into queue
- void put(FlowFileRecord *flow);
- //! Poll the flow file from queue, the expired flow file record also
being returned
- FlowFileRecord *poll(std::set &expiredFlowRecords);
- //! Drain the flow records
- void drain();
+class Connection : public core::Connectable,
+public std::enable_shared_from_this {
+ public:
+ //! Constructor
+ /*!
+ * Create a new processor
+ */
+ explicit Connection(std::string name, uuid_t uuid = NU
[GitHub] nifi-minifi-cpp pull request #59: MINIFI-224 - fixes for locating ZLIB on Ub...
Github user jdye64 closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/59 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #59: MINIFI-224 - fixes for locating ZLIB on Ubuntu 16...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/59 @apiri The environment where I was experiencing these issues was a niche environment really outside of the normal setup. Also I have found a better way to handle this situation in that specific environment. I'm going to close this PR as I do not think it is needed any longer. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #61: MINIFI-229 - Remove third party tests from projec...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/61 @apiri yes this was my intention. I have removed it from the LICENSE in this commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #63: MINIFI-217: Implement namespaces.
Github user jdye64 commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/63
@phrocker when compiling I'm seeing a couple of warnings that look like
`
Serializable.cpp:190:21: warning: comparison of constant 2047 with
expression of type 'char' is always false
} else if (c > 2047){`
Any thoughts on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #65: MINIFI-235: Resolve hostname ident
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/65 Tested using official MiNiFi Docker image against official NiFi Docker image running on Mac OS X host. Was able to successfully transfer flowfiles from minifi to nifi using fqdn where previously only ip address worked. Thanks @randerzander for pointing that out and @phrocker for the quick fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #65: MINIFI-235: Resolve hostname ident
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/65 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #64: MINIFI-235: Use the appropriate connection when r...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/64 @phrocker looks good. Thank you for helping out with this issue! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #64: MINIFI-235: Use the appropriate connection when r...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/64 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #58: MINIFI-145 - Updating Travis to perform builds on...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/58 Looks good Aldrin. Thanks for adding this was needed for sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #61: MINIFI-229 - Remove third party tests from...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/61 MINIFI-229 - Remove third party tests from project You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-229 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/61.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #61 commit 3843adb9a380caa5e95dad7cdbeea9c64d5d8f10 Author: Jeremy Dyer Date: 2017-03-01T22:19:58Z MINIFI-229 - Remove third party tests from project --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #929: NIFI-2613 - Apache POI processor to convert Excel documents...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/929 @jvwing the ```ConvertAvroToParquet``` had slipped my mind thanks for the reminder there. The POM references had been fixed in a previous fix at https://github.com/apache/nifi/pull/929/commits/474a08eda8497d47468e511e3642766995dad62a#diff-e7171c7f07ed58f51e4643dc340432f8R22 I'm unsure what your preference for reviewing is but let me know if this is getting to the point where you would like for me to squash the commits or not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #59: MINIFI-224 - fixes for locating ZLIB on Ub...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/59 MINIFI-224 - fixes for locating ZLIB on Ubuntu 16.10 devices You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-224 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/59.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #59 commit 1caaa63643e2031e2a373dea9280434d56f8a798 Author: Jeremy Dyer Date: 2017-02-25T18:50:00Z MINIFI-224 - fixes for locating ZLIB on Ubuntu 16.10 devices --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #37: MINIFI-171 Dynamic Properties support for process...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/37 @apiri so do you think we should just close this PR then? Trying to tidy up a few PRs this morning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #929: NIFI-2613 - Apache POI processor to convert Excel documents...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/929 @jvwing I indeed had used a much earlier version of the processor. I couldn't find the other version so just ended up doing a rewrite for the most part. Thanks for hanging with me on this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #37: MINIFI-171 Dynamic Properties support for process...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/37 @apiri Where do you think we stand on this? I know you had some concerns about the implementation approach in the beginning. Is that still the case? If so what can we do to resolve those? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #54: MINIFI-206: Implement Extensible Comm Mech...
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/54#discussion_r102845154
--- Diff: libminifi/CMakeLists.txt ---
@@ -36,12 +36,23 @@ ENDIF(POLICY CMP0048)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
+include(CheckCXXCompilerFlag)
+CHECK_CXX_COMPILER_FLAG("-std=c++11" COMPILER_SUPPORTS_CXX11)
+CHECK_CXX_COMPILER_FLAG("-std=c++0x" COMPILER_SUPPORTS_CXX0X)
+if(COMPILER_SUPPORTS_CXX11)
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -g")
+elseif(COMPILER_SUPPORTS_CXX0X)
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")
+else()
+ message(STATUS "The compiler ${CMAKE_CXX_COMPILER} has no C++11 support.
Please use a different C++ compiler.")
+endif()
+
include_directories(../include)
include_directories(../thirdparty/yaml-cpp-yaml-cpp-0.5.3/include)
include_directories(../thirdparty/civetweb-1.9.1/include)
include_directories(include)
-file(GLOB SOURCES "src/*.cpp")
+file(GLOB_RECURSE SOURCES "src/*.cpp")
--- End diff --
Will think conflict with conditional builds like in
https://github.com/apache/nifi-minifi-cpp/pull/52/files#diff-2672ba1ce10995865ff39a95c67bc5dcR72
since those features are in fact subdirectories of src?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102844031
--- Diff: libminifi/test/unit/gps/GetGPSTests.h ---
@@ -0,0 +1,37 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+#ifndef GPS_TESTS
+#define GPS_TESTS
+#include "../../TestBase.h"
+
+#include "../ProvenanceTestHelper.h"
+#include "../Provenance.h"
+#include "../FlowFileRecord.h"
+
+
+TEST_CASE("Test GPSD Create", "[TestGPSEventRecord]"){
--- End diff --
@apiri and @phrocker I'm leaning on you guys for recommendations around the
tests here. I was planning on using http://www.catb.org/gpsd/gpsfake.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #38: MINIFI-175 Create official Apache NiFi MiNiFi C++...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/38 @apiri I have made some changes. This should solve our problems with building and running the docker image. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102765412
--- Diff: libminifi/src/FlowController.cpp ---
@@ -289,6 +289,12 @@ Processor
*FlowControllerImpl::createProcessor(std::string name, uuid_t uuid) {
return NULL;
}
+ #ifdef BUILD_GPS
--- End diff --
This was not very smart of me ... this will cause an error to be thrown
that the processor of type GetGPS does not exist at runtime because the last
else condition will be triggered before this. I need to move this higher up the
condition before the last else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102764886 --- Diff: README.md --- @@ -110,6 +118,9 @@ $ brew install cmake \ leveldb \ ossp-uuid \ boost \ openssl + +# If building with GPS support +$ brew install gps --- End diff -- Its actually is just gps for OS X. Its the gps.h headers and such that is needed. However it might make sense to also install gpsd so I'll add that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102758892
--- Diff: libminifi/test/unit/gps/GetGPSTests.h ---
@@ -0,0 +1,37 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+#ifndef GPS_TESTS
+#define GPS_TESTS
+#include "../../TestBase.h"
+
+#include "../ProvenanceTestHelper.h"
+#include "../Provenance.h"
+#include "../FlowFileRecord.h"
+
+
+TEST_CASE("Test GPSD Create", "[TestGPSEventRecord]"){
--- End diff --
certainly. I'm wondering the best way to do this at the moment. I want to
use gpsfake for this but thinking through the best way to do so. Using gpsfake
would require another system dependency since it is an extra GPSD service
itself.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user jdye64 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102294797
--- Diff: libminifi/include/GetGPS.h ---
@@ -0,0 +1,73 @@
+/**
+ * @file GetGPS.h
+ * GetGPS class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __GET_GPS_H__
+#define __GET_GPS_H__
+
+#include "FlowFileRecord.h"
+#include "Processor.h"
+#include "ProcessSession.h"
+
+//! GetGPS Class
+class GetGPS : public Processor
+{
+public:
+ //! Constructor
+ /*!
+* Create a new processor
+*/
+ GetGPS(std::string name, uuid_t uuid = NULL)
+ : Processor(name, uuid)
+ {
+ _logger = Logger::getLogger();
+ _gpsdHost = "localhost";
+ _gpsdPort = "2947";
+ _gpsdWaitTime = 5000;
+ }
+ //! Destructor
+ virtual ~GetGPS()
+ {
+ }
+ //! Processor Name
+ static const std::string ProcessorName;
+ //! Supported Properties
+ static Property GPSDHost;
+ static Property GPSDPort;
+ static Property GPSDWaitTime;
+
+ //! Supported Relationships
+ static Relationship Success;
+
+public:
+ //! OnTrigger method, implemented by NiFi GetGPS
+ virtual void onTrigger(ProcessContext *context, ProcessSession
*session);
+ //! Initialize, over write by NiFi GetGPS
+ virtual void initialize(void);
+
+protected:
+
+private:
+ //! Logger
+ Logger *_logger;
--- End diff --
No, I agree with you about this. I'll change them to make them follow the
google guidelines. Best to start those new habits now =)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #42: MINIFI-186 Updating Copyright statements to refle...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/42 https://git-wip-us.apache.org/repos/asf?p=nifi-minifi-cpp.git;a=commit;h=9f1c2f438628281bad4323ba554ed5b01f137b00 closes #42 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #38: MINIFI-175 Create official Apache NiFi MiNiFi C++...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/38 @apiri I think we are good for a first pass on the Docker image in my opinion. I think after we use it in the wild a couple of times we will have a better understanding of how people want to use it an can adapt as necessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/52 Support for GPSD integration You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-218 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/52.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #52 commit c020d33f63659ed4695d65af7af478808dd9c3e1 Author: Jeremy Dyer Date: 2017-02-21T05:28:24Z Support for GPSD integration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #929: NIFI-2613 - Apache POI processor to convert Excel documents...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/929 @jvwing thank you for pointing this out. Let me look it over right now and I'll make another commit shortly. I had this in another project and I think during the copy/paste I have made an error. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #37: MINIFI-171 Dynamic Properties support for process...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/37 @phrocker thanks a ton for the suggestions. Updated with your recommendations. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #929: NIFI-2613 - Apache POI processor to convert Excel documents...
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/929 @jvwing wow sorry its been a long time for my response. I've tried to go through and make all of the cleanup suggestions you have made. Yes, the intention is a single output per sheet in the excel file. Keep in mind that if the sheet is not .xlsx and rather .xls format you would see the behavior you are experiencing. Can you attach the excel doc you were testing with? I made the change to case of the attribute Also adjusted the error handling to prevent the end user from being pummeled with a full stack trace. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #397: NIFI-1815
Github user jdye64 closed the pull request at: https://github.com/apache/nifi/pull/397 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #397: NIFI-1815
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/397 Unfortunately this isn't going to work due to license issues so closing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #397: NIFI-1815
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/397 @trixpan unfortunately I was unable to get this implementation to run properly when removing the libraries that had licensing conflicts. Its unfortunate but maybe we just need to close this issue since I don't think its going to work due to license issues. Thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1414: NIFI-3343
Github user jdye64 closed the pull request at: https://github.com/apache/nifi/pull/1414 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1414: NIFI-3343
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1414 @mcgilman that makes perfect sense. Thanks for pointing me to to code snippet. I'll close this PR and the JIRA --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #37: MINIFI-171 Dynamic Properties support for process...
Github user jdye64 commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/37 @apiri I agree. That was my intent was to allow each Processor itself to determine how/if it used the dynamic properties that was the reason for creating the new set in Processor.cpp for the dynamic properties instead of just lumping them all into the same "supportedProperties" set. The only reason for the framework "switch" was just since the CPP version is meant to run on embedded and lightweight hardware maybe the guard against preventing someone from flooding the yml flow with so many dynamic properties that the embedded device ran out of memory and crashed. I see your point and in my own shallow mind I thought that was what I was trying to do but maybe your right and this implementation doesn't make sense? What were you thinking? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1414: NIFI-3343
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi/pull/1414 NIFI-3343 Annotate client dto fields with JsonFormat annotation to allow default Jackson ObjectMapper implementation to be able to deserialize JSON. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi NIFI-3343 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1414.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1414 commit 45151dfdc85291071eae53f2445eeaf03a0052ee Author: Jeremy Dyer Date: 2017-01-13T17:32:29Z NIFI-3343 Annotate client dto fields with JsonFormat annotation to allow default Jackson ObjectMapper implementation to be able to deserialize JSON. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1372#discussion_r95209790 --- Diff: nifi-docker/dockerhub/DockerBuild.sh --- @@ -0,0 +1,31 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#!/bin/bash + +DOCKER_UID=1000 +if [ -n "$1" ]; then + DOCKER_UID="$1" +fi + +DOCKER_GID=50 +if [ -n "$2" ]; then + DOCKER_GID="$2" +fi + +DOCKER_IMAGE="$(egrep -v '(^#|^\s*$|^\s*\t*#)' DockerImage.txt)" +NIFI_IMAGE_VERSION="$(echo "apachenifi:1.1.1" | cut -d : -f 2)" --- End diff -- it certainly should. I must have just overlooked that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1372#discussion_r94867503 --- Diff: nifi-docker/docker/Dockerfile --- @@ -0,0 +1,47 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +FROM java:8u91-jdk +MAINTAINER Apache NiFi + +ARG UID +ARG GID +ARG NIFI_VERSION +ARG NIFI_BINARY + +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION + +# Setup NiFi user +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` +RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi +RUN mkdir -p $NIFI_HOME + +ADD $NIFI_BINARY $NIFI_BASE_DIR --- End diff -- I did a little research and your right. It will have to download the binary since the built binary would not be referenceable. I don't think it would be the end of the world to have 2 Dockerfiles. Really the files could be identically aside from the piece that pulls in the binary. This one could be used for people doing local builds and development while the one that downloads the binary could be used for pushes to Dockerhub. I'll work something up for that here shortly and make another commit. Thanks everyone for chiming in on this. There are a lot of ways to do this I know and things keep changing a little bit but we will get there =) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-site pull request #10: NIFI-3285
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-site/pull/10 NIFI-3285 Updated website committers list to include Jeremy and Joey You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-site NIFI-3285 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-site/pull/10.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #10 commit 0bcf312172da6d6ddf14fd374c749913dfad0fc8 Author: Jeremy Dyer Date: 2017-01-05T16:08:49Z NIFI-3285 Updated website committers list to include Jeremy and Joey --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #38: MINIFI-175 Create official Apache NiFi MiN...
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/38 MINIFI-175 Create official Apache NiFi MiNiFi C++ Dockerfile Created version 1 of official NiFi-MiNiFi-CPP docker image. The image can be built by running a custom target âmake dockerâ please note that âmake packageâ must first be ran to ensure the binary assembly is present. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-minifi-cpp MINIFI-175 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/38.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #38 commit da3ca8468383fe497a7a8b658964f2b2c1c91dde Author: Jeremy Dyer Date: 2017-01-05T02:51:44Z MINIFI-175 Create official Apache NiFi MiNiFi C++ Dockerfile Created version 1 of official NiFi-MiNiFi-CPP docker image. The image can be built by running a custom target âmake dockerâ please note that âmake packageâ must first be ran to ensure the binary assembly is present. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 Refactored to a single Dockerfile and also incorporated the docker-maven-plugin for building the docker image. You can build the image now by invoking the profile "docker" Ex: mvn clean install package -Pdocker from the $NIFI_HOME directory. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 @achristianson wow. I saw the io.fabric8 one but over the spotify one. Thanks for pointing that out --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 @achristianson is the docker-maven-plugin in a public maven repository somewhere that you know of? I don't see it in maven central. I built locally and have it working but we would need it somewhere public. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 @apiri I didn't realize that the tags alone were sufficient without the directories but after reading a little deeper it seems like your right. Let me remove the directories and consolidate to a single Dockerfile and make another commit then we can continue discussion from there. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 Good point about Docker ARG. If the plugin doesn't currently support I'm sure it would be an easy add. Sorry I wasn't very clear there. I was thinking about a more infrastructure level. For us to actually host our official image on Dockerhub there must be a unique tag, commit, and directory that points to our Dockerfile for a specific Docker tag. So if we had a minor release we would need to create a new directory. That is why in this commit I have the two directories "1.1.0" and "1.1.1" to illustrate that. I don't like that structure at all but it almost seems necessary to host via Dockerhub. This is somewhat of a limitation for "official" images only which most likely other users of the docker-maven-plugin are not doing but rather making internal release and sending to their own private docker repos. So when we made a PR to https://github.com/docker-library/official-images our PR would be a single file looking something like ... https://github.com/docker-library/official-images/blob/master/library/storm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1372: NIFI-3260 Official Docker Image
Github user jdye64 commented on the issue: https://github.com/apache/nifi/pull/1372 @achristianson I have mixed emotions about incorporating this into the build process but I am ok with it if that is what most people think. My original thought was that this would be a more singular build that we as a community release for end users to download. However from this discussion it seems like most people might be more interested in the build process approach. In that case it should be pretty simple to do a little maven magic to inject the current version into the Dockerfile and use the docker-maven-plugin. I do believe this might make maintenance on older Docker images a little harder since they would be tightly coupled to a release/branch however. Does anyone more seasoned than myself have ideas about how to make that process easier? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
