[jira] [Resolved] (NIFI-6309) PutSFTP does not validate that privatekey needs a key passphrase
[ https://issues.apache.org/jira/browse/NIFI-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura resolved NIFI-6309. - Resolution: Not A Bug Thanks for reporting the issue [~scotthowell]. However, I believe this is an expected behavior. Unencrypted RSA private key doesn't have passphrase. Although unencrypted RSA private key is less secure since it can be used once it's stollen, it's commonly used especially in dev-environments. We should allow a private key path without a passphrase. > PutSFTP does not validate that privatekey needs a key passphrase > > > Key: NIFI-6309 > URL: https://issues.apache.org/jira/browse/NIFI-6309 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration Management >Affects Versions: 1.9.2 > Environment: amazon-linux-2 >Reporter: Scott Howell >Priority: Minor > > When using a private key with PutSFTP the code requires that there is a key > passphrase also used. The processor does not show invalidated once you add > the private key. Would be helpful to have some validation on the processor to > show that you also need to include a key passphrase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] alopresto commented on issue #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on issue #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi… URL: https://github.com/apache/nifi/pull/3482#issuecomment-494644304 Thanks Nathan. I ran this locally and it behaved well; and all the tests pass. I think there are still some outstanding requests on `JwtAuthenticationFilter` and `NiFiAuthenticationFilter`. I also rebased this against current master and everything looked good with the exception of an unrelated checkstyle error introduced elsewhere (I fixed that and some other minor issues in [0939f95](https://github.com/apache/nifi/commit/0939f951a3de776f74fa9f50d0a75a11a6f46b9d)). Feel free to pull that commit and make the above changes, and I'll merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286303213
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/test/groovy/org/apache/nifi/web/security/jwt/JwtAuthenticationFilterTest.groovy
##
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.security.jwt;
+
+import net.minidev.json.JSONObject;
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass
+import org.junit.Rule;
+import org.junit.Test
+import org.junit.rules.ExpectedException;
+import org.junit.runner.RunWith;
+import org.junit.runners.JUnit4;
+
+@RunWith(JUnit4.class)
+class JwtAuthenticationFilterTest extends GroovyTestCase {
+
+public static String jwtString;
+
+@Rule
+public ExpectedException expectedException = ExpectedException.none();
+
+@BeforeClass
+public static void setUpOnce() throws Exception {
+final String ALG_HEADER = "{\"alg\":\"HS256\"}"
+final int EXPIRATION_SECONDS = 500
+Calendar now = Calendar.getInstance()
+final long currentTime = (long) (now.getTimeInMillis() / 1000.0)
+final long TOKEN_ISSUED_AT = currentTime
+final long TOKEN_EXPIRATION_SECONDS = currentTime + EXPIRATION_SECONDS
+
+// Generate a token that we will add a valid signature from a
different token
+// Always use LinkedHashMap to enforce order of the keys because the
signature depends on order
+Map claims = new LinkedHashMap<>()
+claims.put("sub", "unknownuser")
+claims.put("iss", "MockIdentityProvider")
+claims.put("aud", "MockIdentityProvider")
+claims.put("preferred_username", "unknownuser")
+claims.put("kid", 1)
+claims.put("exp", TOKEN_EXPIRATION_SECONDS)
+claims.put("iat", TOKEN_ISSUED_AT)
+final String EXPECTED_PAYLOAD = new JSONObject(claims).toString()
Review comment:
The `net.minidev.json` import is still present but unused. I'll clean it up
on merge.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286302980
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtAuthenticationFilter.java
##
@@ -43,15 +46,27 @@ public Authentication attemptAuthentication(final
HttpServletRequest request) {
// TODO: Refactor request header extraction logic to shared utility as
it is duplicated in AccessResource
// get the principal out of the user token
-final String authorization = request.getHeader(AUTHORIZATION);
+final String authentication = request.getHeader(AUTHORIZATION);
-// if there is no authorization header, we don't know the user
-if (authorization == null || !StringUtils.startsWith(authorization,
BEARER)) {
+// if there is no authorization (authentication) header, we don't know
the user
+if (authentication == null || !validJwtFormat(authentication)) {
return null;
} else {
// Extract the Base64 encoded token from the Authorization header
-final String token = StringUtils.substringAfterLast(authorization,
" ");
+final String token = getTokenFromHeader(authentication);
return new JwtAuthenticationRequestToken(token,
request.getRemoteAddr());
}
}
+
+private boolean validJwtFormat(String authenticationHeader) {
+Matcher matcher = tokenPattern.matcher(authenticationHeader);
+return matcher.matches();
+}
+
+private String getTokenFromHeader(String authenticationHeader) {
+Matcher matcher = tokenPattern.matcher(authenticationHeader);
Review comment:
Same.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286302944
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtAuthenticationFilter.java
##
@@ -43,15 +46,27 @@ public Authentication attemptAuthentication(final
HttpServletRequest request) {
// TODO: Refactor request header extraction logic to shared utility as
it is duplicated in AccessResource
// get the principal out of the user token
-final String authorization = request.getHeader(AUTHORIZATION);
+final String authentication = request.getHeader(AUTHORIZATION);
-// if there is no authorization header, we don't know the user
-if (authorization == null || !StringUtils.startsWith(authorization,
BEARER)) {
+// if there is no authorization (authentication) header, we don't know
the user
+if (authentication == null || !validJwtFormat(authentication)) {
return null;
} else {
// Extract the Base64 encoded token from the Authorization header
-final String token = StringUtils.substringAfterLast(authorization,
" ");
+final String token = getTokenFromHeader(authentication);
return new JwtAuthenticationRequestToken(token,
request.getRemoteAddr());
}
}
+
+private boolean validJwtFormat(String authenticationHeader) {
Review comment:
I believe this request is still outstanding.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286288057
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/paths/RecordPathCompiler.java
##
@@ -237,14 +240,16 @@ public static RecordPathSegment buildPath(final Tree
tree, final RecordPathSegme
return new ReplaceNull(args[0], args[1], absolute);
}
case "concat": {
-final int numArgs = argumentListTree.getChildCount();
-
-final RecordPathSegment[] argPaths = new
RecordPathSegment[numArgs];
-for (int i = 0; i < numArgs; i++) {
-argPaths[i] =
buildPath(argumentListTree.getChild(i), null, absolute);
-}
-
-return new Concat(argPaths, absolute);
+return new
Concat(getArgumentsForStringFunction(absolute, argumentListTree), absolute);
+}
+case "toLowerCase": {
+return new
ToLowerCase(getArgumentsForStringFunction(absolute, argumentListTree),
absolute);
+}
+case "toUpperCase": {
+return new
ToUpperCase(getArgumentsForStringFunction(absolute, argumentListTree),
absolute);
+}
+case "trim": {
+return new
TrimString(getArgumentsForStringFunction(absolute, argumentListTree), absolute);
Review comment:
Please add these new functions to 'Standalone Functions' section of
'record-path-guide.adoc'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak edited a comment on issue #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak edited a comment on issue #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper… URL: https://github.com/apache/nifi/pull/3478#issuecomment-494622338 Moved this comment to [here](https://github.com/apache/nifi/pull/3478#discussion_r286287474) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286287474
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/AbstractStringFunction.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.StandardFieldValue;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.util.stream.Stream;
+
+public abstract class AbstractStringFunction extends RecordPathSegment {
Review comment:
To explain why we need to return multiple converted FieldValues stream with
`trim`, `toUpperCase` or `toLowerCase`, I suggest adding following test case:
```java
@Test
public void testTrimArray() {
final List fields = new ArrayList<>();
final DataType dataType = new
ArrayDataType(RecordFieldType.STRING.getDataType());
fields.add(new RecordField("names", dataType));
final RecordSchema schema = new SimpleRecordSchema(fields);
final Map values = new HashMap<>();
values.put("names", new String[]{" John Smith ", " Jane
Smith "});
final Record record = new MapRecord(schema, values);
final List results =
RecordPath.compile("trim(/names[*])").evaluate(record).getSelectedFields().collect(Collectors.toList());
assertEquals("John Smith", results.get(0).getValue());
assertEquals("Jane Smith", results.get(1).getValue());
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on issue #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on issue #3478: NIFI-6304 added trim, toLowerCase and
toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#issuecomment-494622338
To explain why we need to return multiple converted FieldValues stream with
`trim`, `toUpperCase` or `toLowerCase`, I suggest adding following test case:
```java
@Test
public void testTrimArray() {
final List fields = new ArrayList<>();
final DataType dataType = new
ArrayDataType(RecordFieldType.STRING.getDataType());
fields.add(new RecordField("names", dataType));
final RecordSchema schema = new SimpleRecordSchema(fields);
final Map values = new HashMap<>();
values.put("names", new String[]{" John Smith ", " Jane
Smith "});
final Record record = new MapRecord(schema, values);
final List results =
RecordPath.compile("trim(/names[*])").evaluate(record).getSelectedFields().collect(Collectors.toList());
assertEquals("John Smith", results.get(0).getValue());
assertEquals("Jane Smith", results.get(1).getValue());
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286281428
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/ToLowerCase.java
##
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.util.stream.Stream;
+
+public class ToLowerCase extends AbstractStringFunction {
+public ToLowerCase(RecordPathSegment[] valuePaths, boolean absolute) {
+super("lowercase", valuePaths, absolute);
+}
+
+@Override
+public Stream evaluate(RecordPathEvaluationContext context) {
+return super.evaluate(context, stream -> {
+StringBuilder sb = new StringBuilder();
+stream.forEach(fv ->
sb.append(DataTypeUtils.toString(fv.getValue(), (String) null).toLowerCase()));
Review comment:
Possible NPE.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286283431
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/AbstractStringFunction.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.StandardFieldValue;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.util.stream.Stream;
+
+public abstract class AbstractStringFunction extends RecordPathSegment {
+protected final RecordPathSegment[] valuePaths;
+protected String pathValue;
+
+public AbstractStringFunction(final String path, final RecordPathSegment[]
valuePaths, final boolean absolute) {
+super(path, null, absolute);
+this.valuePaths = valuePaths;
+this.pathValue = path;
+}
+
+public Stream evaluate(RecordPathEvaluationContext context,
EvaluationCallback callback) {
+Stream evaluated = Stream.empty();
+
+for (final RecordPathSegment valuePath : valuePaths) {
+final Stream stream = valuePath.evaluate(context);
+evaluated = Stream.concat(evaluated, stream);
+}
Review comment:
Although the original code in Concat.java uses this approach, I'd writ as
follows instead:
```
final Stream evaluated =
Arrays.stream(valuePaths).flatMap(valuePath -> valuePath.evaluate(context));
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286281335
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/ToUpperCase.java
##
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.util.stream.Stream;
+
+public class ToUpperCase extends AbstractStringFunction {
+final RecordPathSegment[] valuePaths;
+public static final String PATH_VALUE = "uppercase";
+
+public ToUpperCase(final RecordPathSegment[] valuePaths, final boolean
absolute) {
+super(PATH_VALUE, valuePaths, absolute);
+this.valuePaths = valuePaths;
+}
+
+@Override
+public Stream evaluate(RecordPathEvaluationContext context) {
+return super.evaluate(context, stream -> {
+StringBuilder sb = new StringBuilder();
+stream.forEach(fv ->
sb.append(DataTypeUtils.toString(fv.getValue(), (String) null).toUpperCase()));
Review comment:
Possible NPE.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286284222
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/AbstractStringFunction.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.StandardFieldValue;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.util.stream.Stream;
+
+public abstract class AbstractStringFunction extends RecordPathSegment {
Review comment:
I don't think this abstraction is the right approach. Because:
- `Concat` returns a stream with only one concatenated value using
`Stream.of(responseValue)`
- Ex. {'a', 'b', 'c'} =concat=> {'abc'}
- But `trim`, `toLowerCase` and `toUpperCase` should return a stream
containing all processed results of incoming field values.
- Ex. {'a', 'b', 'c'} =toUpperCase=> {'A', 'B', 'C'}
- Current PR code produces {'a', 'b', 'c'} =toUpperCase=> {'ABC'}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim, toLowerCase and toUpperCase to record path oper…
ijokarumawak commented on a change in pull request #3478: NIFI-6304 added trim,
toLowerCase and toUpperCase to record path oper…
URL: https://github.com/apache/nifi/pull/3478#discussion_r286281007
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/TrimString.java
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.util.stream.Stream;
+
+public class TrimString extends AbstractStringFunction {
+public TrimString(RecordPathSegment[] valuePaths, boolean absolute) {
+super("trim", valuePaths, absolute);
+}
+
+@Override
+public Stream evaluate(RecordPathEvaluationContext context) {
+return super.evaluate(context, stream -> {
+StringBuilder sb = new StringBuilder();
+stream.forEach(fv ->
sb.append(DataTypeUtils.toString(fv.getValue(), (String) null).trim()));
Review comment:
If the field value is null, this will cause NPE.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS - Added try/catch to protect against secondary exception in a…
ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS
- Added try/catch to protect against secondary exception in a…
URL: https://github.com/apache/nifi/pull/3476#discussion_r286277585
##
File path:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
##
@@ -126,7 +123,11 @@ public Void doInJms(final Session session) throws
JMSException {
// We need to call recover to ensure that in the event of
// abrupt end or exception the current session will stop
message
// delivery and restart with the oldest unacknowledged
message
-session.recover();
+try {
+session.recover();
+} catch (Exception e1) {
+// likely the session is closed...just ignore and
press on with initial problem
Review comment:
I assume the intent of caching the exception `e1` here is throwing the
original exception `e` instead so that users can see the root cause of the
issue. Is that correct?
If so, a comment describing that will help other devs understanding the
purpose later.
Also, please add a debug or trace level log message using `e1` here, instead
of completely ignoring it. So that users can change processor's log level to
see what caused recovery exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS - Added try/catch to protect against secondary exception in a…
ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS
- Added try/catch to protect against secondary exception in a…
URL: https://github.com/apache/nifi/pull/3476#discussion_r286276474
##
File path:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
##
@@ -126,7 +123,11 @@ public Void doInJms(final Session session) throws
JMSException {
// We need to call recover to ensure that in the event of
// abrupt end or exception the current session will stop
message
// delivery and restart with the oldest unacknowledged
message
-session.recover();
+try {
+session.recover();
+} catch (Exception e1) {
+// likely the session is closed...just ignore and
press on with initial problem
+}
Review comment:
This is the only change required to address NIFI-6022. Please keep original
source code as much as possible for easier review cycles.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS - Added try/catch to protect against secondary exception in a…
ijokarumawak commented on a change in pull request #3476: NIFI-6022 ConsumeJMS
- Added try/catch to protect against secondary exception in a…
URL: https://github.com/apache/nifi/pull/3476#discussion_r286277585
##
File path:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
##
@@ -126,7 +123,11 @@ public Void doInJms(final Session session) throws
JMSException {
// We need to call recover to ensure that in the event of
// abrupt end or exception the current session will stop
message
// delivery and restart with the oldest unacknowledged
message
-session.recover();
+try {
+session.recover();
+} catch (Exception e1) {
+// likely the session is closed...just ignore and
press on with initial problem
Review comment:
I assume the intent of catching the exception `e1` here is throwing the
original exception `e` instead so that users can see the root cause of the
issue. Is that correct?
If so, a comment describing that will help other devs understanding the
purpose later.
Also, please add a debug or trace level log message using `e1` here, instead
of completely ignoring it. So that users can change processor's log level to
see what caused recovery exception.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6022) ConsumeJMS - admin yielding
[ https://issues.apache.org/jira/browse/NIFI-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845392#comment-16845392 ] Koji Kawamura commented on NIFI-6022: - Hi [~slyouts]. Thanks for reporting this, and providing the PR. I've added 'contributor' role to your JIRA account so that you can assign yourself to NiFi JIRAs. (I've assigned you this time) BTW, the intent of this fix is to throwing the original exception instead of the one thrown when we call 'session.recover()'. Is that correct? If so, more descriptive JIRA title would be appreciated. For example, "ConsumeJMS should throw the original exception when session.recover fails". I'll post few review comments on the PR as well. Thanks! > ConsumeJMS - admin yielding > --- > > Key: NIFI-6022 > URL: https://issues.apache.org/jira/browse/NIFI-6022 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.7.1 >Reporter: Steven Youtsey >Assignee: Steven Youtsey >Priority: Major > Labels: easyfix > Time Spent: 10m > Remaining Estimate: 0h > > Processor admin yields when session with JMS provider is closed. > When an exception occurs (no idea what as it was stepped on) and the session > is closed, the exception handler attempts to use the session and another > exception is thrown and not caught by the processor. See JMSConsumer, line > 129. Need to wrap that with a try/catch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-6022) ConsumeJMS - admin yielding
[ https://issues.apache.org/jira/browse/NIFI-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-6022: --- Assignee: Steven Youtsey > ConsumeJMS - admin yielding > --- > > Key: NIFI-6022 > URL: https://issues.apache.org/jira/browse/NIFI-6022 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.7.1 >Reporter: Steven Youtsey >Assignee: Steven Youtsey >Priority: Major > Labels: easyfix > Time Spent: 10m > Remaining Estimate: 0h > > Processor admin yields when session with JMS provider is closed. > When an exception occurs (no idea what as it was stepped on) and the session > is closed, the exception handler attempts to use the session and another > exception is thrown and not caught by the processor. See JMSConsumer, line > 129. Need to wrap that with a try/catch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-874) Disable AWS extension
[ https://issues.apache.org/jira/browse/MINIFICPP-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-874. -- Resolution: Fixed > Disable AWS extension > - > > Key: MINIFICPP-874 > URL: https://issues.apache.org/jira/browse/MINIFICPP-874 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #565: MINIFICPP-874: Remove AWS from travis
asfgit closed pull request #565: MINIFICPP-874: Remove AWS from travis URL: https://github.com/apache/nifi-minifi-cpp/pull/565 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] jdye64 commented on issue #565: MINIFICPP-874: Remove AWS from travis
jdye64 commented on issue #565: MINIFICPP-874: Remove AWS from travis URL: https://github.com/apache/nifi-minifi-cpp/pull/565#issuecomment-494601088 +1 yeah this makes a lot of sense This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] phrocker opened a new pull request #565: MINIFICPP-874: Remove AWS from travis
phrocker opened a new pull request #565: MINIFICPP-874: Remove AWS from travis URL: https://github.com/apache/nifi-minifi-cpp/pull/565 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (MINIFICPP-874) Disable AWS extension
Mr TheSegfault created MINIFICPP-874: Summary: Disable AWS extension Key: MINIFICPP-874 URL: https://issues.apache.org/jira/browse/MINIFICPP-874 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Mr TheSegfault Assignee: Mr TheSegfault -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-6310) SyslogReader does not support octet counting
[ https://issues.apache.org/jira/browse/NIFI-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845254#comment-16845254 ] Otto Fowler commented on NIFI-6310: --- Since the RFC6587 is for syslog over tcp, and is a format 'above' 5424 and 3164 I don't think this is a SyslogReader issue. This would be something that ListenSyslog would support instead, since it reads off the wire. But I may be wrong [~bbende]? What other over the wire syslog ingest would we have? Where do you think this would belong? > SyslogReader does not support octet counting > > > Key: NIFI-6310 > URL: https://issues.apache.org/jira/browse/NIFI-6310 > Project: Apache NiFi > Issue Type: New Feature >Affects Versions: 1.9.2 >Reporter: Anand Parmar >Priority: Minor > > If a syslog is sent in a octet counted format, then the SyslogReader cannot > read it. SyslogReader assumes that the syslog is newline delimited. > Octet counting is described in > https://tools.ietf.org/html/rfc6587#section-3.4.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #557: MINIFICPP-850: Add operating system to c2 response
asfgit closed pull request #557: MINIFICPP-850: Add operating system to c2 response URL: https://github.com/apache/nifi-minifi-cpp/pull/557 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6310) SyslogReader does not support octet counting
Anand Parmar created NIFI-6310: -- Summary: SyslogReader does not support octet counting Key: NIFI-6310 URL: https://issues.apache.org/jira/browse/NIFI-6310 Project: Apache NiFi Issue Type: New Feature Affects Versions: 1.9.2 Reporter: Anand Parmar If a syslog is sent in a octet counted format, then the SyslogReader cannot read it. SyslogReader assumes that the syslog is newline delimited. Octet counting is described in https://tools.ietf.org/html/rfc6587#section-3.4.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] AaronLeon commented on a change in pull request #3410: NIFI-6100 Use setBytes in JdbcCommon for binary types
AaronLeon commented on a change in pull request #3410: NIFI-6100 Use setBytes
in JdbcCommon for binary types
URL: https://github.com/apache/nifi/pull/3410#discussion_r286171830
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java
##
@@ -807,7 +807,7 @@ public static void setParameter(final PreparedStatement
stmt, final String attrN
throw new ParseException("Unable to parse binary
data using the formatter `" + valueFormat + "`.",0);
}
-stmt.setBinaryStream(parameterIndex, new
ByteArrayInputStream(bValue), bValue.length);
+stmt.setBytes(parameterIndex, bValue);
Review comment:
Hmm.. what about adding an upstream check to see if PreparedStatement
instanceof PhoenixPreparedStatement, which can be handled independently? Using
try/catch as control flow doesn't seem quite right, esp. for something that
supports operations that need to be very performant (e.g. data ingestion). This
would give us a place to handle other JDBC drivers that are not totally
compliant and handle the case where developers wish to support two drivers that
are incompatible (for example, one that only has .setBytes(...) _and_ one that
only has .setBinaryStream(...)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] AaronLeon commented on a change in pull request #3410: NIFI-6100 Use setBytes in JdbcCommon for binary types
AaronLeon commented on a change in pull request #3410: NIFI-6100 Use setBytes
in JdbcCommon for binary types
URL: https://github.com/apache/nifi/pull/3410#discussion_r286171830
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java
##
@@ -807,7 +807,7 @@ public static void setParameter(final PreparedStatement
stmt, final String attrN
throw new ParseException("Unable to parse binary
data using the formatter `" + valueFormat + "`.",0);
}
-stmt.setBinaryStream(parameterIndex, new
ByteArrayInputStream(bValue), bValue.length);
+stmt.setBytes(parameterIndex, bValue);
Review comment:
Hmm.. what about adding an upstream check to see if PreparedStatement
instanceof PhoenixPreparedStatement, which can be handled independently? Using
try/catch as control flow doesn't seem quite right, esp. for something that
supports operations that need to be very performant (e.g. data ingestion). This
would also give us a pattern to handle other JDBC drivers that are not totally
compliant and handle the case where developers wish to support two drivers that
are incompatible (for example, one that only has .setBytes(...) _and_ one that
only has .setBinaryStream(...)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (MINIFICPP-615) Provide Windows OpenSSL Support
[ https://issues.apache.org/jira/browse/MINIFICPP-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-615: - Resolution: Fixed Status: Resolved (was: Patch Available) > Provide Windows OpenSSL Support > --- > > Key: MINIFICPP-615 > URL: https://issues.apache.org/jira/browse/MINIFICPP-615 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug > Environment: Microsoft Windows >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: windows > Fix For: 0.7.0 > > > Ensure OpenSSL support works throughout our project in windows. > > MINIFICPP-6 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-10) Invalid UUID does not throw sensical error
[ https://issues.apache.org/jira/browse/MINIFICPP-10?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-10: --- Assignee: Arpad Boda > Invalid UUID does not throw sensical error > --- > > Key: MINIFICPP-10 > URL: https://issues.apache.org/jira/browse/MINIFICPP-10 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Trivial > Labels: easyfix, newbie > Fix For: 0.7.0 > > > If a UUID is invalid for any component, the program will seg fault with an > exception: > "Could not locate a source with id %s to create a connection" > This should be made more clear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-820) Facilitate C2 work for operations
[ https://issues.apache.org/jira/browse/MINIFICPP-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-820: Assignee: Mr TheSegfault > Facilitate C2 work for operations > - > > Key: MINIFICPP-820 > URL: https://issues.apache.org/jira/browse/MINIFICPP-820 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Blocker > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-778) Docker build and test directories should be copied into build directory, not source tree
[ https://issues.apache.org/jira/browse/MINIFICPP-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-778: Assignee: Mr TheSegfault > Docker build and test directories should be copied into build directory, not > source tree > > > Key: MINIFICPP-778 > URL: https://issues.apache.org/jira/browse/MINIFICPP-778 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-778) Docker build and test directories should be copied into build directory, not source tree
[ https://issues.apache.org/jira/browse/MINIFICPP-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-778: Assignee: Arpad Boda (was: Mr TheSegfault) > Docker build and test directories should be copied into build directory, not > source tree > > > Key: MINIFICPP-778 > URL: https://issues.apache.org/jira/browse/MINIFICPP-778 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-620) Test/Fix CAPI on windows
[ https://issues.apache.org/jira/browse/MINIFICPP-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-620: Assignee: Alex Marmer > Test/Fix CAPI on windows > > > Key: MINIFICPP-620 > URL: https://issues.apache.org/jira/browse/MINIFICPP-620 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature > Environment: Microsoft Windows >Reporter: Mr TheSegfault >Assignee: Alex Marmer >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.7.0 > > > Ensure that the C API examples work on windows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-873) Create windows event collector
Mr TheSegfault created MINIFICPP-873: Summary: Create windows event collector Key: MINIFICPP-873 URL: https://issues.apache.org/jira/browse/MINIFICPP-873 Project: Apache NiFi MiNiFi C++ Issue Type: Brainstorming Reporter: Mr TheSegfault Assignee: Alex Marmer Fix For: 0.7.0 Create windows event collector – place holder. [~amarmer] please define the details here -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-782) Better define deprecated macros
[ https://issues.apache.org/jira/browse/MINIFICPP-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-782: Assignee: Arpad Boda > Better define deprecated macros > --- > > Key: MINIFICPP-782 > URL: https://issues.apache.org/jira/browse/MINIFICPP-782 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-836) Validate stop and notifystop allow processors to flush
[ https://issues.apache.org/jira/browse/MINIFICPP-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-836: Assignee: Mr TheSegfault > Validate stop and notifystop allow processors to flush > -- > > Key: MINIFICPP-836 > URL: https://issues.apache.org/jira/browse/MINIFICPP-836 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Blocker > Labels: ReloadValidation > Fix For: 0.7.0 > > > In the event that a flow update occurs,validate that notify stop allows all > processors to complete and flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-733) librdkafka logs go to stdout
[ https://issues.apache.org/jira/browse/MINIFICPP-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-733: Assignee: Daniel Bakai > librdkafka logs go to stdout > > > Key: MINIFICPP-733 > URL: https://issues.apache.org/jira/browse/MINIFICPP-733 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Daniel Bakai >Priority: Major > Fix For: 0.7.0 > > > Currently all the librdkafka logs go to stdout which makes it both annoying > in a terminal window and difficult to debug since the logs are in stdout. We > should change this so that librdkafka logs go to the same minifi-app.log that > already exists and is configuration via the minifi-log.properties file -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-834) Create static links for lzma
[ https://issues.apache.org/jira/browse/MINIFICPP-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-834: Assignee: Arpad Boda > Create static links for lzma > > > Key: MINIFICPP-834 > URL: https://issues.apache.org/jira/browse/MINIFICPP-834 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > Original Estimate: 72h > Remaining Estimate: 72h > > Statically link lzma when shared libs are turned off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-780) Change GenerateFlowFile to allow 0b content FlowFIles
[ https://issues.apache.org/jira/browse/MINIFICPP-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-780: Assignee: Mr TheSegfault > Change GenerateFlowFile to allow 0b content FlowFIles > - > > Key: MINIFICPP-780 > URL: https://issues.apache.org/jira/browse/MINIFICPP-780 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > > Would also behoove us to fill some gaps with this processor since it's used > so frequently -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-800) Add ability to docker env to load all processors.
[ https://issues.apache.org/jira/browse/MINIFICPP-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-800: Assignee: Daniel Bakai > Add ability to docker env to load all processors. > -- > > Key: MINIFICPP-800 > URL: https://issues.apache.org/jira/browse/MINIFICPP-800 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Major > Fix For: 0.7.0 > > > Currently processors are defined with the default constructors. We should > augment this in the test environment so that we can use a similar path that > Python processors do to support better python testing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-30) TailFile should support globs/wildcards
[ https://issues.apache.org/jira/browse/MINIFICPP-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-30: --- Assignee: Mr TheSegfault > TailFile should support globs/wildcards > --- > > Key: MINIFICPP-30 > URL: https://issues.apache.org/jira/browse/MINIFICPP-30 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Andre F de Miranda >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > > Similarly to NIFI-1170, it would be great if MINIFI Java/CPP tail supported > wildcards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-30) TailFile should support globs/wildcards
[ https://issues.apache.org/jira/browse/MINIFICPP-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-30: --- Assignee: Arpad Boda (was: Mr TheSegfault) > TailFile should support globs/wildcards > --- > > Key: MINIFICPP-30 > URL: https://issues.apache.org/jira/browse/MINIFICPP-30 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Andre F de Miranda >Assignee: Arpad Boda >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > > Similarly to NIFI-1170, it would be great if MINIFI Java/CPP tail supported > wildcards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-721) Add time in queue/repo
[ https://issues.apache.org/jira/browse/MINIFICPP-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-721: Assignee: Mr TheSegfault > Add time in queue/repo > -- > > Key: MINIFICPP-721 > URL: https://issues.apache.org/jira/browse/MINIFICPP-721 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > > Add metrics for time in queue to better debug various parameters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-660) Create docker test for python library
[ https://issues.apache.org/jira/browse/MINIFICPP-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-660: Assignee: Arpad Boda > Create docker test for python library > - > > Key: MINIFICPP-660 > URL: https://issues.apache.org/jira/browse/MINIFICPP-660 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-810) add docker verify to travis
[ https://issues.apache.org/jira/browse/MINIFICPP-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-810: Assignee: Daniel Bakai > add docker verify to travis > --- > > Key: MINIFICPP-810 > URL: https://issues.apache.org/jira/browse/MINIFICPP-810 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-815) Add FileUtils Tests
[ https://issues.apache.org/jira/browse/MINIFICPP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-815: Assignee: Arpad Boda > Add FileUtils Tests > --- > > Key: MINIFICPP-815 > URL: https://issues.apache.org/jira/browse/MINIFICPP-815 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Formerly titled: > [Dog fooded test should use a separate test function to validate the lamdba > function|https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > [ > https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > is a dog fooded test. > > We should probably make a FileUtils suite of tests as a focal point than > specifically addressing the above class's tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-814) Add more ListenHTTP tests
[ https://issues.apache.org/jira/browse/MINIFICPP-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-814: Assignee: Daniel Bakai > Add more ListenHTTP tests > - > > Key: MINIFICPP-814 > URL: https://issues.apache.org/jira/browse/MINIFICPP-814 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Dustin Rodrigues >Assignee: Daniel Bakai >Priority: Major > Fix For: 0.7.0 > > > Add ListenHTTP tests to verify that SSL Verify Peer and Authorized DN Pattern > options work as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-866) GetTCPTests should use random port
[ https://issues.apache.org/jira/browse/MINIFICPP-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-866. -- Resolution: Fixed > GetTCPTests should use random port > -- > > Key: MINIFICPP-866 > URL: https://issues.apache.org/jira/browse/MINIFICPP-866 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.7.0 > > Time Spent: 20m > Remaining Estimate: 0h > > GetTCPTests now use 3 hardcoded ports, which might cause collisions with > local services. > Ports should be randomly selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #564: MINIFICPP-866 - GetTCPTests should use random port
asfgit closed pull request #564: MINIFICPP-866 - GetTCPTests should use random port URL: https://github.com/apache/nifi-minifi-cpp/pull/564 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6100) JdbcCommon does not support binary types for Phoenix
[ https://issues.apache.org/jira/browse/NIFI-6100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-6100: --- Resolution: Fixed Fix Version/s: 1.10.0 Status: Resolved (was: Patch Available) > JdbcCommon does not support binary types for Phoenix > - > > Key: NIFI-6100 > URL: https://issues.apache.org/jira/browse/NIFI-6100 > Project: Apache NiFi > Issue Type: Bug >Reporter: Aaron Leon >Priority: Critical > Fix For: 1.10.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The current implementation uses java.sql.PreparedStatement.setBinaryStream, > which throws SQLFeatureNotSupportedException when used with > org.apache.phoenix.jdbc.PhoenixPreparedStatement. A better alternative is to > use setBytes. There will be no performance hit since the binary value is > already read into memory in the current implementation, and it is implemented > in PhoenixPreparedStatement so this will fix compatibility with Phoenix. It > is also noted that PhoenixPreparedStatement should probably implement > setBinaryStream anyways, so ideally a PR will be sent to Phoenix as well. > https://github.com/apache/nifi/blob/rel/nifi-1.7.0/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L832 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-6100) JdbcCommon does not support binary types for Phoenix
[ https://issues.apache.org/jira/browse/NIFI-6100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845116#comment-16845116 ] ASF subversion and git services commented on NIFI-6100: --- Commit 81ddd02ca89cc068a6367eaeae575bac5c5e2ad5 in nifi's branch refs/heads/master from Aaron Leon [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=81ddd02 ] NIFI-6100 Use setBytes in JdbcCommon for binary types This closes #3410 Signed-off-by: Mike Thomsen > JdbcCommon does not support binary types for Phoenix > - > > Key: NIFI-6100 > URL: https://issues.apache.org/jira/browse/NIFI-6100 > Project: Apache NiFi > Issue Type: Bug >Reporter: Aaron Leon >Priority: Critical > Time Spent: 40m > Remaining Estimate: 0h > > The current implementation uses java.sql.PreparedStatement.setBinaryStream, > which throws SQLFeatureNotSupportedException when used with > org.apache.phoenix.jdbc.PhoenixPreparedStatement. A better alternative is to > use setBytes. There will be no performance hit since the binary value is > already read into memory in the current implementation, and it is implemented > in PhoenixPreparedStatement so this will fix compatibility with Phoenix. It > is also noted that PhoenixPreparedStatement should probably implement > setBinaryStream anyways, so ideally a PR will be sent to Phoenix as well. > https://github.com/apache/nifi/blob/rel/nifi-1.7.0/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L832 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] asfgit closed pull request #3410: NIFI-6100 Use setBytes in JdbcCommon for binary types
asfgit closed pull request #3410: NIFI-6100 Use setBytes in JdbcCommon for binary types URL: https://github.com/apache/nifi/pull/3410 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] MikeThomsen commented on a change in pull request #3410: NIFI-6100 Use setBytes in JdbcCommon for binary types
MikeThomsen commented on a change in pull request #3410: NIFI-6100 Use setBytes
in JdbcCommon for binary types
URL: https://github.com/apache/nifi/pull/3410#discussion_r286160733
##
File path:
nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java
##
@@ -807,7 +807,7 @@ public static void setParameter(final PreparedStatement
stmt, final String attrN
throw new ParseException("Unable to parse binary
data using the formatter `" + valueFormat + "`.",0);
}
-stmt.setBinaryStream(parameterIndex, new
ByteArrayInputStream(bValue), bValue.length);
+stmt.setBytes(parameterIndex, bValue);
Review comment:
@mattyb149 @patricker I'm not a fan of setting a flag because after skimming
over that method I didn't see a good place to store a state to track whether
the except was thrown in a previous pass. I'll add the try/catch and merge.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] bbende commented on issue #3455: NIFI-5900 Add SelectJson processor
bbende commented on issue #3455: NIFI-5900 Add SelectJson processor URL: https://github.com/apache/nifi/pull/3455#issuecomment-494496040 I think Matty B and Mark are the most involved with the record related processors, so they might be able to offer the best guidance. From a high-level, it seems like maybe we should have both approaches (this PR and Mike's streaming reader). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #561: MINIFICPP-854 CaptureRTSPFrame with OpenCV
asfgit closed pull request #561: MINIFICPP-854 CaptureRTSPFrame with OpenCV URL: https://github.com/apache/nifi-minifi-cpp/pull/561 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (MINIFICPP-865) Statically link AWS SDK to libminifi
[ https://issues.apache.org/jira/browse/MINIFICPP-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-865: Assignee: Arpad Boda (was: Mr TheSegfault) > Statically link AWS SDK to libminifi > > > Key: MINIFICPP-865 > URL: https://issues.apache.org/jira/browse/MINIFICPP-865 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Users should have the ability to choose if the AWS SDK is dynamically or > statically linked. Today when we build it can only be statically linked. > However its likely the best way forward is having it always be statically > linked -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-865) Statically link AWS SDK to libminifi
[ https://issues.apache.org/jira/browse/MINIFICPP-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845057#comment-16845057 ] Mr TheSegfault commented on MINIFICPP-865: -- [~aboda] I know you're working OPC-UA, but any way you can take a look at this too? Thanks! > Statically link AWS SDK to libminifi > > > Key: MINIFICPP-865 > URL: https://issues.apache.org/jira/browse/MINIFICPP-865 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Users should have the ability to choose if the AWS SDK is dynamically or > statically linked. Today when we build it can only be statically linked. > However its likely the best way forward is having it always be statically > linked -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the
matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286134486
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/ProxiedEntitiesUtils.java
##
@@ -148,13 +147,13 @@ public static String
buildProxiedEntitiesChainString(final NiFiUser user) {
return StringUtils.join(proxyChain, "");
}
-public static void successfulAuthorization(HttpServletRequest request,
HttpServletResponse response, Authentication authResult) {
+public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
Review comment:
Made a mistake with that first comment:
```
/**
* If a successfully authenticated request was made via a proxy, relevant
proxy headers will be added to the response.
*
* @param request The proxied client request that was successfully
authenticated.
* @param response A servlet response to the client containing the successful
authentication details.
*/
public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the
matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286134486
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/ProxiedEntitiesUtils.java
##
@@ -148,13 +147,13 @@ public static String
buildProxiedEntitiesChainString(final NiFiUser user) {
return StringUtils.join(proxyChain, "");
}
-public static void successfulAuthorization(HttpServletRequest request,
HttpServletResponse response, Authentication authResult) {
+public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
Review comment:
Made a mistake with that first comment:
/**
* If a successfully authenticated request was made via a proxy, relevant
proxy headers will be added to the response.
*
* @param request The proxied client request that was successfully
authenticated.
* @param response A servlet response to the client containing the successful
authentication details.
*/
public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ottobackwards removed a comment on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows
ottobackwards removed a comment on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows URL: https://github.com/apache/nifi/pull/3241#issuecomment-494473014 So no on the documentation then or writeup then eh? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (MINIFICPP-866) GetTCPTests should use random port
[ https://issues.apache.org/jira/browse/MINIFICPP-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845028#comment-16845028 ] Mr TheSegfault commented on MINIFICPP-866: -- [~aboda] I love these changes. Thanks! Can we target 0.8.0 for the remainder of changes like these? > GetTCPTests should use random port > -- > > Key: MINIFICPP-866 > URL: https://issues.apache.org/jira/browse/MINIFICPP-866 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.7.0 > > Time Spent: 10m > Remaining Estimate: 0h > > GetTCPTests now use 3 hardcoded ports, which might cause collisions with > local services. > Ports should be randomly selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] ottobackwards commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows
ottobackwards commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows URL: https://github.com/apache/nifi/pull/3241#issuecomment-494473014 So no on the documentation then or writeup then eh? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (NIFIREG-272) Release manager activites for 0.4.0
[ https://issues.apache.org/jira/browse/NIFIREG-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFIREG-272. - Resolution: Fixed > Release manager activites for 0.4.0 > --- > > Key: NIFIREG-272 > URL: https://issues.apache.org/jira/browse/NIFIREG-272 > Project: NiFi Registry > Issue Type: Task >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Fix For: 0.4.0 > > > Perform release manager duties for 0.4.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286116510
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/ProxiedEntitiesUtils.java
##
@@ -148,13 +147,13 @@ public static String
buildProxiedEntitiesChainString(final NiFiUser user) {
return StringUtils.join(proxyChain, "");
}
-public static void successfulAuthorization(HttpServletRequest request,
HttpServletResponse response, Authentication authResult) {
+public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
Review comment:
I don't understand the `@param request The original client request that
failed to be authenticated` for `successfulAuthentication` -- do you mean that
the incoming request wasn't authenticated because it came from the proxy? I am
looking for method comments that explain _why_ and _when_ this method gets
called.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the
matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286114915
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtAuthenticationFilter.java
##
@@ -43,15 +46,27 @@ public Authentication attemptAuthentication(final
HttpServletRequest request) {
// TODO: Refactor request header extraction logic to shared utility as
it is duplicated in AccessResource
// get the principal out of the user token
-final String authorization = request.getHeader(AUTHORIZATION);
+final String authentication = request.getHeader(AUTHORIZATION);
Review comment:
I have named it authorizationHeader :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286114844
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/AccessResource.java
##
@@ -766,13 +766,17 @@ public Response logOut(@Context HttpServletRequest
httpServletRequest, @Context
throw new IllegalStateException("User authentication/authorization
is only supported when running over HTTPS.");
}
-String authorizationHeader =
httpServletRequest.getHeader(JwtAuthenticationFilter.AUTHORIZATION);
-final String token =
StringUtils.substringAfterLast(authorizationHeader, " ");
-try {
-jwtService.logOut(token);
-return generateOkResponse().build();
-} catch (final JwtException e) {
-return Response.serverError().build();
+String userIdentity = NiFiUserUtils.getNiFiUserIdentity();
+
+if(userIdentity != null && !userIdentity.isEmpty()) {
+try {
+jwtService.logOut(userIdentity);
+return generateOkResponse().build();
+} catch (final JwtException e) {
+return Response.serverError().build();
Review comment:
Sorry, I meant that even if the eventual error was logged somewhere in a
stacktrace, I would expect a line written here in the `nifi-user.log` file
showing the failure to logout. Just add `USER_FILE` appender, aka
`nifi-user.log`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
alopresto commented on a change in pull request #3482: NIFI-6280 - Broke out
the matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286113307
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/jwt/JwtAuthenticationFilter.java
##
@@ -43,15 +46,27 @@ public Authentication attemptAuthentication(final
HttpServletRequest request) {
// TODO: Refactor request header extraction logic to shared utility as
it is duplicated in AccessResource
// get the principal out of the user token
-final String authorization = request.getHeader(AUTHORIZATION);
+final String authentication = request.getHeader(AUTHORIZATION);
Review comment:
I agree the HTTP header name is misleading, but since that header is called
"Authorization", I think we should be consistent. You could name it
`authorizationHeaderContainingAuthenticationCredentials` to be explicit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] SamHjelmfelt commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows
SamHjelmfelt commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows URL: https://github.com/apache/nifi/pull/3241#issuecomment-494455533 Merged! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] SamHjelmfelt closed pull request #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows
SamHjelmfelt closed pull request #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows URL: https://github.com/apache/nifi/pull/3241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] markap14 commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows
markap14 commented on issue #3241: NIFI-5922: Stateless NiFi, an alternative runtime for NiFi flows URL: https://github.com/apache/nifi/pull/3241#issuecomment-494449289 @SamHjelmfelt I know this has taken a while to get everything resolved, but it's a great contribution! Many thanks for sticking with it until we got it all sorted out. I've now merged to master! There's still plenty to do here, of course, but I've run several flows with it, and I think it's in good enough shape to go ahead and merge. Thanks again! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844983#comment-16844983 ] ASF subversion and git services commented on NIFI-5922: --- Commit 8245bc3f804f701da52a4e64a000b04aff718948 in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8245bc3 ] NIFI-5922: Ensure that we import any default variable values on flow import > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844982#comment-16844982 ] ASF subversion and git services commented on NIFI-5922: --- Commit 60b8fcac1ba4dff64c28139ddf282d915f4c221b in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=60b8fca ] NIFI-5922: Bug fixes; initialize, setup, and enable controller services; code cleanup > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844980#comment-16844980 ] ASF subversion and git services commented on NIFI-5922: --- Commit cbafd8f834813f4e23c961a282a6f44b5a62c078 in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=cbafd8f ] NIFI-5922: Addressed checkstyle violations; added to README Updates to nifi-fn proposal: - Separated into nifi-fn-core, nifi-fn-bootstrap, nifi-fn-nar, nifi-fn-assembly; fully unpacks nars and runs flows - Rebased against master and updated to version 1.10.0-SNAPSHOT - Removed dependency on nifi-framework-core - Added LICENSE/NOTICE files > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844979#comment-16844979 ] ASF subversion and git services commented on NIFI-5922: --- Commit 417b3955d796857209000f497b817268a7cf3bdb in nifi's branch refs/heads/master from Sam Hjelmfelt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=417b395 ] NIFI-5922: First Commit for NiFi-Stateless > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844981#comment-16844981 ] ASF subversion and git services commented on NIFI-5922: --- Commit 146689b36fb75bb1b14f408a9b7771505215ca4e in nifi's branch refs/heads/master from samhjelmfelt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=146689b ] NIFI-5922: Renaming from NiFi-Fn to NiFi-Stateless Fixed docker image and moved it into the nifi-docker project Fixed Docker container, YARN runtime, and OpenWhisk runtime > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-5922) Stateless NiFi, an alternative runtime for NiFi flows
[ https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-5922. -- Resolution: Fixed Fix Version/s: 1.10.0 > Stateless NiFi, an alternative runtime for NiFi flows > - > > Key: NIFI-5922 > URL: https://issues.apache.org/jira/browse/NIFI-5922 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sam Hjelmfelt >Priority: Major > Fix For: 1.10.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > NiFi-Fn is a library for running NiFi flows as stateless functions. It > provides similar delivery guarantees as NiFi without the need for on-disk > repositories by waiting to confirm receipt ofincoming data until it has been > written to the destination. This is similar to Storm’s acking mechanism and > Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this > is completely handled by the framework while still supporting all NiFi > processors and controller services natively without change.This results in > the ability to run NiFi flows as ephemeral, stateless functions and should be > able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and > scalability while leveraging the vast library of NiFi processors and the NiFi > UI for building custom flows. > By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn > flows can be deployed that take full advantage of the platform’s scale and > multi-tenancy features. By leveraging Function as a Service engines (FaaS) > (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event > sources (or just cron) for event-driven data movement where flows only run > when triggered and pricing is measured at the 100ms granularity. By combining > the two, large-scale batch processing could also be performed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the
matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286092332
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/ProxiedEntitiesUtils.java
##
@@ -148,13 +147,13 @@ public static String
buildProxiedEntitiesChainString(final NiFiUser user) {
return StringUtils.join(proxyChain, "");
}
-public static void successfulAuthorization(HttpServletRequest request,
HttpServletResponse response, Authentication authResult) {
+public static void successfulAuthentication(HttpServletRequest request,
HttpServletResponse response) {
Review comment:
Are these correct?:
```
/**
* If using a proxy, tell the proxy the proxied user identity was
authenticated.
*
* @param request The original client request that failed to be authenticated.
* @param response Servlet response to the client containing the successful
authentication details.
*/
```
/**
* If using a proxy, give the proxy an explanation why authentication
failed.
*
* @param request The proxied client request that failed to be
authenticated.
* @param response Servlet response to the proxied client containing the
unsuccessful authentication attempt details.
* @param failed The related exception thrown and explanation for the
unsuccessful authentication attempt.
*/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (MINIFICPP-872) CopyAzureBlob Processor
Jeremy Dyer created MINIFICPP-872: - Summary: CopyAzureBlob Processor Key: MINIFICPP-872 URL: https://issues.apache.org/jira/browse/MINIFICPP-872 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Processor for copying an Azure blob object asynchronously using the Azure Python SDK [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#copy-blob-container-name--blob-name--copy-source--metadata-none--source-if-modified-since-none--source-if-unmodified-since-none--source-if-match-none--source-if-none-match-none--destination-if-modified-since-none--destination-if-unmodified-since-none--destination-if-match-none--destination-if-none-match-none--destination-lease-id-none--source-lease-id-none--timeout-none-] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-871) GetAzureBlobToStream Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Dyer updated MINIFICPP-871: -- Description: Get an Azure blob object and stream the contents of that object into a new flowfile using the Azure Python SDK method [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#get-blob-to-stream-container-name--blob-name--stream--snapshot-none--start-range-none--end-range-none--validate-content-false--progress-callback-none--max-connections-2--lease-id-none--if-modified-since-none--if-unmodified-since-none--if-match-none--if-none-match-none--timeout-none-] was:Get an Azure blob object and stream the contents of that object into a new flowfile > GetAzureBlobToStream Processor > -- > > Key: MINIFICPP-871 > URL: https://issues.apache.org/jira/browse/MINIFICPP-871 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.7.0 > > > Get an Azure blob object and stream the contents of that object into a new > flowfile using the Azure Python SDK method > [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#get-blob-to-stream-container-name--blob-name--stream--snapshot-none--start-range-none--end-range-none--validate-content-false--progress-callback-none--max-connections-2--lease-id-none--if-modified-since-none--if-unmodified-since-none--if-match-none--if-none-match-none--timeout-none-] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-871) GetAzureBlobToStream Processor
Jeremy Dyer created MINIFICPP-871: - Summary: GetAzureBlobToStream Processor Key: MINIFICPP-871 URL: https://issues.apache.org/jira/browse/MINIFICPP-871 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Get an Azure blob object and stream the contents of that object into a new flowfile -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-870) DeleteAzureBlob Processor
Jeremy Dyer created MINIFICPP-870: - Summary: DeleteAzureBlob Processor Key: MINIFICPP-870 URL: https://issues.apache.org/jira/browse/MINIFICPP-870 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Create a processor for deleting an Azure blob object using the python SDK method [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#delete-blob-container-name--blob-name--snapshot-none--lease-id-none--delete-snapshots-none--if-modified-since-none--if-unmodified-since-none--if-match-none--if-none-match-none--timeout-none-] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-869) ListAzureBlobs Processor
Jeremy Dyer created MINIFICPP-869: - Summary: ListAzureBlobs Processor Key: MINIFICPP-869 URL: https://issues.apache.org/jira/browse/MINIFICPP-869 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Create a new processor to list the blobs that are available in Azure blob storage using the python SDK method. [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#list-blobs-container-name--prefix-none--num-results-none--include-none--delimiter-none--marker-none--timeout-none-] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-868) Create AzureCredentialsService
Jeremy Dyer created MINIFICPP-868: - Summary: Create AzureCredentialsService Key: MINIFICPP-868 URL: https://issues.apache.org/jira/browse/MINIFICPP-868 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Controller service for managing the Azure credentials. This service will house the credentials in a secure manner and proxy the credentials to the configured processor when requested. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-867) Provide an implementation for all Blob storage aspects of the Azure
[ https://issues.apache.org/jira/browse/MINIFICPP-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Dyer updated MINIFICPP-867: -- Description: Provide an implementation for utilizing Azure Blob storage. Azure currently does not have a C++ SDK. However they do have a good python library. Since MiNifi C++ now supports native python processors it seems logical that we offer out of the box processors and controller services so that we can Put, Get, List, Fetch, Copy, and Delete objects in Azure blob storage. (was: Provide an implementation for fully utilizing all aspects of the AWS C++ SDK. This includes downloading and building the SDK locally as part of the build and then implementing processors for the major S3 operations such as GetObject, ListObject, FetchObject, PutObject, DeleteObject, and CopyObject. As part of this epic the work should include a CredentialsController service that allows for centralized configuration of the AWS credentials so they need not be provided directly in the processors themselves.) > Provide an implementation for all Blob storage aspects of the Azure > --- > > Key: MINIFICPP-867 > URL: https://issues.apache.org/jira/browse/MINIFICPP-867 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.7.0 > > > Provide an implementation for utilizing Azure Blob storage. Azure currently > does not have a C++ SDK. However they do have a good python library. Since > MiNifi C++ now supports native python processors it seems logical that we > offer out of the box processors and controller services so that we can Put, > Get, List, Fetch, Copy, and Delete objects in Azure blob storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-867) Provide an implementation for all Blob storage aspects of the Azure
Jeremy Dyer created MINIFICPP-867: - Summary: Provide an implementation for all Blob storage aspects of the Azure Key: MINIFICPP-867 URL: https://issues.apache.org/jira/browse/MINIFICPP-867 Project: Apache NiFi MiNiFi C++ Issue Type: Epic Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.7.0 Provide an implementation for fully utilizing all aspects of the AWS C++ SDK. This includes downloading and building the SDK locally as part of the build and then implementing processors for the major S3 operations such as GetObject, ListObject, FetchObject, PutObject, DeleteObject, and CopyObject. As part of this epic the work should include a CredentialsController service that allows for centralized configuration of the AWS credentials so they need not be provided directly in the processors themselves. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-867) Provide an implementation for all Blob storage aspects of the Azure
[ https://issues.apache.org/jira/browse/MINIFICPP-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Dyer updated MINIFICPP-867: -- Epic Name: Azure Blob Storage Integration (was: AWS S3 Integration) > Provide an implementation for all Blob storage aspects of the Azure > --- > > Key: MINIFICPP-867 > URL: https://issues.apache.org/jira/browse/MINIFICPP-867 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.7.0 > > > Provide an implementation for fully utilizing all aspects of the AWS C++ SDK. > This includes downloading and building the SDK locally as part of the build > and then implementing processors for the major S3 operations such as > GetObject, ListObject, FetchObject, PutObject, DeleteObject, and CopyObject. > As part of this epic the work should include a CredentialsController service > that allows for centralized configuration of the AWS credentials so they need > not be provided directly in the processors themselves. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi-minifi-cpp] arpadboda opened a new pull request #564: MINIFICPP-866 - GetTCPTests should use random port
arpadboda opened a new pull request #564: MINIFICPP-866 - GetTCPTests should use random port URL: https://github.com/apache/nifi-minifi-cpp/pull/564 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (MINIFICPP-866) GetTCPTests should use random port
Arpad Boda created MINIFICPP-866: Summary: GetTCPTests should use random port Key: MINIFICPP-866 URL: https://issues.apache.org/jira/browse/MINIFICPP-866 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.6.0 Reporter: Arpad Boda Assignee: Arpad Boda Fix For: 0.7.0 GetTCPTests now use 3 hardcoded ports, which might cause collisions with local services. Ports should be randomly selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-550) Create RocksDB Controller Service
[ https://issues.apache.org/jira/browse/MINIFICPP-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844888#comment-16844888 ] Mr TheSegfault commented on MINIFICPP-550: -- The original description states that site to site is the end goal, but I think this is something we need accessible in all processors/controller services. don't know if this means shared state or not. > Create RocksDB Controller Service > - > > Key: MINIFICPP-550 > URL: https://issues.apache.org/jira/browse/MINIFICPP-550 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > > A RocksDB Controller service will give us the ability to store arbitrary > information into controller services that can later be sent via SiteToSite. > This will support many of my monitoring and test use cases. Using RocksDB as > a key/value store we can serialize and send this information periodically -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-550) Create RocksDB Controller Service
[ https://issues.apache.org/jira/browse/MINIFICPP-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844886#comment-16844886 ] Mr TheSegfault commented on MINIFICPP-550: -- [~bakaid] This ticket oversimplifies what is needed, but there is a concept of a variable registry internally that allows EL to access data. Well, there is the need to store arbitrary data – I think you ran into this with dynamic properties. This may be something that will also be used by metrics ( if it's performant enough ). > Create RocksDB Controller Service > - > > Key: MINIFICPP-550 > URL: https://issues.apache.org/jira/browse/MINIFICPP-550 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > > A RocksDB Controller service will give us the ability to store arbitrary > information into controller services that can later be sent via SiteToSite. > This will support many of my monitoring and test use cases. Using RocksDB as > a key/value store we can serialize and send this information periodically -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix memleaks
bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix
memleaks
URL: https://github.com/apache/nifi-minifi-cpp/pull/563#discussion_r286042867
##
File path: libminifi/src/core/Processor.cpp
##
@@ -176,7 +176,7 @@ void
Processor::removeConnection(std::shared_ptr conn) {
// Connection is destination to the current processor
if (_incomingConnections.find(connection) != _incomingConnections.end()) {
_incomingConnections.erase(connection);
- connection->setDestination(NULL);
Review comment:
I prefer nullptr too, but setDestination now gets a weak_ptr and weak_ptr
has no constructor from nullptr_t (or any pointer for that matter).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] asfgit closed pull request #562: MINIFICPP-856 Introduce AWS SDK to the MiNiFi Cmake build
asfgit closed pull request #562: MINIFICPP-856 Introduce AWS SDK to the MiNiFi Cmake build URL: https://github.com/apache/nifi-minifi-cpp/pull/562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix memleaks
bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix
memleaks
URL: https://github.com/apache/nifi-minifi-cpp/pull/563#discussion_r286041202
##
File path: libminifi/src/core/repository/VolatileContentRepository.cpp
##
@@ -81,10 +81,7 @@ void VolatileContentRepository::start() {
return;
if (running_)
return;
- thread_ = std::thread(::run,
shared_from_parent());
- thread_.detach();
Review comment:
As far as I could gather it served no discernible purpose, but because a
shared_ptr instance is captured in a detached thread it causes valgrind to
issue a warning (as valgrind will evaluate memory leaks at exit()).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix memleaks
bakaid commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix
memleaks
URL: https://github.com/apache/nifi-minifi-cpp/pull/563#discussion_r286041202
##
File path: libminifi/src/core/repository/VolatileContentRepository.cpp
##
@@ -81,10 +81,7 @@ void VolatileContentRepository::start() {
return;
if (running_)
return;
- thread_ = std::thread(::run,
shared_from_parent());
- thread_.detach();
Review comment:
As far as I could gather it served no discernible purpose, but because a
shared_ptr instance is capture in a detached thread it causes valgrind to issue
a warning (as valgrind will evaluate memory leaks at exit()).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (MINIFICPP-839) Multiple memory leaks because of shared_ptr cycles
[ https://issues.apache.org/jira/browse/MINIFICPP-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844861#comment-16844861 ] Mr TheSegfault commented on MINIFICPP-839: -- I don't think we can support doing a singleton at this time based on how we restart, I also don't think that changing everything to weak ptrs will be ideal for safety of data. Can you isolate leaks to only reloads of the agent? Tests are something we can deal with in other ways if we have to. > Multiple memory leaks because of shared_ptr cycles > -- > > Key: MINIFICPP-839 > URL: https://issues.apache.org/jira/browse/MINIFICPP-839 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Daniel Bakai >Priority: Blocker > Labels: ReloadValidation > Fix For: 0.7.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > There seems to be a Connection<->Processor shared_ptr cycle which means that > the Processors never get destructed. > This causes leaks in reloads and in tests. > There are also other cycles causing leaks: > * Connection <-> FlowFile (this seems to be partially mitigated by > Connection::drain(), but not entirely) > * ResourceClaim <-> VolatileContentRepository > * VolatileContentRepository <-> VolatileContentRepository (capturing a > shared_ptr into a thread func and detaching the thread) > * FlowController <- > SchedulingAgent (TimerDrivenSchedulingAgent, > EventDrivenSchedulingAgent, CronDrivenSchedulingAgent) and even > FlowController <- > util::ThreadPool because of this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [nifi] thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the matching for /access/knox/** and /access/oi…
thenatog commented on a change in pull request #3482: NIFI-6280 - Broke out the
matching for /access/knox/** and /access/oi…
URL: https://github.com/apache/nifi/pull/3482#discussion_r286037140
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/AccessResource.java
##
@@ -766,13 +766,17 @@ public Response logOut(@Context HttpServletRequest
httpServletRequest, @Context
throw new IllegalStateException("User authentication/authorization
is only supported when running over HTTPS.");
}
-String authorizationHeader =
httpServletRequest.getHeader(JwtAuthenticationFilter.AUTHORIZATION);
-final String token =
StringUtils.substringAfterLast(authorizationHeader, " ");
-try {
-jwtService.logOut(token);
-return generateOkResponse().build();
-} catch (final JwtException e) {
-return Response.serverError().build();
+String userIdentity = NiFiUserUtils.getNiFiUserIdentity();
+
+if(userIdentity != null && !userIdentity.isEmpty()) {
+try {
+jwtService.logOut(userIdentity);
+return generateOkResponse().build();
+} catch (final JwtException e) {
+return Response.serverError().build();
Review comment:
Do I push this logging into both the nifi-app.log and nifi-user.log by
adding this to the logback.xml?
```
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] phrocker commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix memleaks
phrocker commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix
memleaks
URL: https://github.com/apache/nifi-minifi-cpp/pull/563#discussion_r286036223
##
File path: libminifi/include/ResourceClaim.h
##
@@ -112,7 +122,7 @@ class ResourceClaim : public
std::enable_shared_from_this {
// Full path to the content
std::string _contentFullPath;
- std::shared_ptr> claim_manager_;
Review comment:
hmmm why are we creating a non-owning reference here? We could potentially
lose data in some routes. Seems worrisome. I'll do some testing to validate.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] phrocker commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix memleaks
phrocker commented on a change in pull request #563: WiP: MINIFICPP-839 - Fix
memleaks
URL: https://github.com/apache/nifi-minifi-cpp/pull/563#discussion_r286035591
##
File path: libminifi/include/core/FlowFile.h
##
@@ -254,29 +254,29 @@ class FlowFile : public core::Connectable, public
ReferenceContainer {
* Sets the original connection with a shared pointer.
* @param connection shared connection.
Review comment:
I presume this is because the processor is weak?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
