[GitHub] nifi issue #3009: NIFI-4532 OpenID Connect User Authentication
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/3009 @SarthakSahu Recommend sticking with /bin/sh, as Alpine base images don't usually contain /bin/bash. ---
[GitHub] nifi issue #3031: NIFI-5635 - Description PutEmail properties with multiple ...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/3031 Reviewed, looks great, +1. ---
[GitHub] nifi issue #2951: NIFI-5474: When using Regex Replace with ReplaceText, and ...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2951 Well since I was the judge on the original NIFI-4272 case and didn't catch the side effects, perhaps I shouldn't be the one to hear the appeal. ---
[GitHub] nifi issue #2949: NIFI-3672 Add support for strongly typed message propertie...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2949 Added a commit to include some documentation ---
[GitHub] nifi pull request #2949: NIFI-3672 Add support for strongly typed message pr...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2949 NIFI-3672 Add support for strongly typed message properties in PublishJMS You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-3672 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2949.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2949 commit 66eeb48802317cdff69fe83070d26dac7245294a Author: Mike Moser Date: 2018-08-13T17:40:54Z NIFI-3672 Add support for strongly typed message properties in PublishJMS ---
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2936 +1 from me, though I didn't test this against a real RabbitMQ server. @zenfenan did you want to take another look? ---
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2936 Please do add Port to your PR, if you can, then I think this will also cover NIFI-4723. It looks like PORT_VALIDATOR already supports expression language, so you are good there. ---
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2936 If you are using VARIABLE_REGISTRY to dynamically evaluate the Host, shouldn't you also allow it for Port? ---
[GitHub] nifi issue #2823: NIFI-5350 Add a way to provide arbitrary Java options in s...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2823 Seems like this could be useful, and certainly doesn't hurt. +1 from me. ---
[GitHub] nifi issue #2931: NIFI-3531 Moved session.recover in JMSConsumer to exceptio...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2931 The JIRA ticket mentioned a dev list conversation where Oleg said it was to limit the possibility of message loss in the event of a processor and/or NiFi crash. However, it has proven to cause problems. I have tried to kill -9 NiFi several times while ConsumeJMS is under heavy load and haven't lost a message, though perhaps I've just been lucky. The javadoc of the session.recover method doesn't seem to indicate that it should be called regularly. The decision regarding what to do with this PR might become a risk management one, where the benefits of session.recover as it is now are unknown but the problems associated with it are known and significant. ---
[GitHub] nifi pull request #2931: NIFI-3531 Moved session.recover in JMSConsumer to e...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2931 NIFI-3531 Moved session.recover in JMSConsumer to exceptional situations You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3531 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2931.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2931 commit 8309747889c3187005a0ce4e4c0c11b1e3281e10 Author: Mike Moser Date: 2018-08-01T20:11:35Z NIFI-3531 Moved session.recover in JMSConsumer to exceptional situations ---
[GitHub] nifi issue #2916: NIFI-5451 Added test resources for 128 bit encryption.
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2916 Reviewed and the issue is resolved, +1 will merge. ---
[GitHub] nifi issue #2904: NIFI-5196 Fixed JMS connection leak when uncaught exceptio...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2904 Thank you @pvillard31, I appreciate your time to merge this. ---
[GitHub] nifi pull request #2904: NIFI-5196 Fixed JMS connection leak when uncaught e...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2904 NIFI-5196 Fixed JMS connection leak when uncaught exceptions happen You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-5196 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2904.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2904 commit 4d4ef545448223cf5cd9bceea1b7778fab535480 Author: Mike Moser Date: 2018-07-17T21:20:47Z NIFI-5196 Fixed JMS connection leak when uncaught exceptions happen ---
[GitHub] nifi pull request #2796: NIFI-5275 PostHTTP SocketConfig setup, fixed connec...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2796 NIFI-5275 PostHTTP SocketConfig setup, fixed connection pool when ... using HTTPS, setup idle connection checker, setup request retry handler, improved some exception handling Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-5275 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2796.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2796 commit ec33b97d4cc6744404e498a20da3a4faebc83c59 Author: Mike Moser Date: 2018-06-14T21:15:39Z NIFI-5275 PostHTTP SocketConfig setup, fixed connection pool when using HTTPS, setup idle connection checker, setup request retry handler, improved some exception handling ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 @mattyb149 I think the issue is whether there is a reasonable expectation that a user would loop back a 'failure' relationship, and precedent set in other similar processors. For example, I consider ReplaceText in the same category as processors that modify content such as CompressContent, UnpackContent, and EncryptContent. None of those processors penalize flowfiles sent to failure. In those processors it's not reasonable to expect a failure to correct itself, so it's not reasonable to loop back the failure relationship. I just followed that precedent when modifying ReplaceText for this PR. ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 Yeah, but there's only 1 other processor in the codebase that does data transformation and does penalize on failure. By far most processors that penalize on failure are ingress/egress processors, where it makes more sense. ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 One never knows how a dataflow manager will design their flow, so I was just trying to cover all bases. I don't think a downstream consumer would notice that the failure files were penalized, but if someone did loop their failure relationship, they would certainly notice if a failed file wan't penalized. I can remove the call to penalize. ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 @ottobackwards This post is old but I think still applies. In general, most regular expression engines use recursion for some things. https://stackoverflow.com/questions/7509905/java-lang-stackoverflowerror-while-using-a-regex-to-parse-big-strings ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 The situation I'm trying to catch uses a regular expression that looks legitimate, so can't be caught while validating configuration, but then blows up when the input flowfile causes the Pattern matcher to produce too many matches while using recursive code. I do think this falls into a Category 3 issue. I simplified the code to catch StackOverflowError and comment why the catch exists. ---
[GitHub] nifi issue #2767: NIFI-5274 avoid rollback on uncaught errors in ReplaceText
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2767 I'm not sure I understand what your suggestion is @joewitt. Would you clarify for me? Should I code it to catch StackOverflowError, which is the use case that I need to handle cleanly? The code pattern I used here has worked really well for me in custom processors to avoid yield/rollback on exceptions caused by flowfile content, while maintaining a desired yield/rollback on framework exceptions in other situations. ---
[GitHub] nifi pull request #2767: NIFI-5274 avoid rollback on uncaught errors in Repl...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2767 NIFI-5274 avoid rollback on uncaught errors in ReplaceText Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-5274 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2767.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2767 commit f874890aa27463bb0c6a755062848990ad81931f Author: Mike Moser Date: 2018-06-06T18:25:32Z NIFI-5274 avoid rollback on uncaught errors in ReplaceText ---
[GitHub] nifi issue #2748: NIFI-4272 ReplaceText support multiple captures when el is...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2748 +1 this looks good to me. I will squash and merge. Thanks @ottobackwards! ---
[GitHub] nifi pull request #2748: NIFI-4272 ReplaceText support multiple captures whe...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2748#discussion_r193232920
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestReplaceText.java
---
@@ -1074,12 +1090,11 @@ public void testRegexWithBadCaptureGroup() throws
IOException {
runner.setProperty(ReplaceText.REPLACEMENT_STRATEGY,
ReplaceText.REGEX_REPLACE);
runner.setProperty(ReplaceText.EVALUATION_MODE,
ReplaceText.ENTIRE_TEXT);
+exception.expect(AssertionError.class);
+exception.expectMessage("java.lang.IndexOutOfBoundsException: No
group 1");
--- End diff --
At first I was really concerned about this change to
IndexOutOfBoundsException, because it means the flowfile will rollback and
admin yield the ReplaceText. But if this is simply verifying behavior when the
SEARCH_VALUE is incorrectly configured, then I think this is OK. Can you
confirm? Normally we would prefer a customValidate() method catch this
scenario (REPLACEMENT_VALUE references a capture group that does exist in the
SEARCH_VALUE) but perhaps that isn't viable at validation time.
---
[GitHub] nifi pull request #2748: NIFI-4272 ReplaceText support multiple captures whe...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2748#discussion_r193230207
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestReplaceText.java
---
@@ -81,15 +81,16 @@ public void testSimple() throws IOException {
@Test
public void testWithEscaped$InReplacement() throws IOException {
final TestRunner runner = getRunner();
-runner.setProperty(ReplaceText.SEARCH_VALUE, "(?s:^.*$)");
+//runner.setProperty(ReplaceText.SEARCH_VALUE, "(?s:^.*$)");
+runner.setProperty(ReplaceText.SEARCH_VALUE, "(?s)(^.*$)");
runner.setProperty(ReplaceText.REPLACEMENT_VALUE, "a\\$b");
runner.enqueue("a$a,b,c,d");
runner.run();
runner.assertAllFlowFilesTransferred(ReplaceText.REL_SUCCESS, 1);
final MockFlowFile out =
runner.getFlowFilesForRelationship(ReplaceText.REL_SUCCESS).get(0);
-out.assertContentEquals("a\\$b".getBytes("UTF-8"));
+out.assertContentEquals("a$b".getBytes("UTF-8"));
--- End diff --
Was this is a change in the way that ReplaceText behaves or was this unit
test incorrect before? Is there a way to obtain t he output "a\$b" from
ReplaceText now?
---
[GitHub] nifi pull request #2748: NIFI-4272 ReplaceText support multiple captures whe...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2748#discussion_r193223710
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
---
@@ -79,7 +80,8 @@
@SystemResourceConsideration(resource = SystemResource.MEMORY)
public class ReplaceText extends AbstractProcessor {
-private static Pattern REPLACEMENT_NORMALIZATION_PATTERN =
Pattern.compile("(\\$\\D)");
+private static Pattern QUOTED_GROUP_REF_PATTERN =
Pattern.compile("('\\$\\d+')");
+private static Pattern LITERAL_QUOTED_PATTERN =
Pattern.compile("literal\\(('.*?')\\)",Pattern.DOTALL);
--- End diff --
Expression Language allows single quotes and double quotes to be used
interchangeably. Can this pattern be updated to handle both?
---
[GitHub] nifi issue #2497: NIFI-3599 Allowed back pressure object count and data size...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2497 Hi @mcgilman I made the suggested changes, rebased from master, and force pushed to this branch. What do you think? ---
[GitHub] nifi issue #2559: NIFI-4658 set Maximum Number of Entries to required with d...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2559 I see the latest update, and +1 it looks good. ---
[GitHub] nifi issue #2559: NIFI-4658 set Maximum Number of Entries to required with d...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2559 Hi @markobean, I reviewed this and I have a couple of questions. I'm not sure I understand why the `throw e;` statements were removed from MergeContent. Can you explain or put them back in? Also, in TestMergeContent, I'm not sure why the `testDefragmentOldStyleAttributes()` test was removed. The test still passes with the rest of your PR applied. Was this perhaps part of the changes made before you modified the PR and force pushed? Thanks for looking into this. ---
[GitHub] nifi issue #2557: NIFI-4950 Defining behavior for MergeContent when more tha...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2557 Reviewed and tested; behavior in the presence of a duplicate fragment is fixed; duplicate is sent to failure. I fixed a typo and used FragmentAttributes.FRAGMENT_INDEX enum in place of "fragment.index" strings. +1, merging. Thanks for fixing this @markobean. ---
[GitHub] nifi issue #2497: NIFI-3599 Allowed back pressure object count and data size...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2497 @mcgilma @markap14 I'm not sure I was able to fully grok "add a ConfigDTO which is set on the AboutDTO" but I gave it a shot. Rebased from master and pushed a new commit. Let me know what you think, thanks! ---
[GitHub] nifi issue #2487: NIFI-4774: Allow user to choose which write-ahead log impl...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2487 @mattyb149 and @markap14 with the 1.6.0 release approaching, do you think we've allowed enough time for review and testing? Is this good to go? Thanks. ---
[GitHub] nifi issue #1240: NIFI-3039 Provenance Repository - Fix PurgeOldEvent and Ro...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1240 I tested this a lot, and based on the conversation in the JIRA ticket, I adjusted the purge low water mark to 88%. This allows the prov_repo to purge old events quicker after a large spike in events, whereas before it could take a long time to bring the prov_repo back down to 90% usage. It also doesn't delete too many events. I also adjusted the rollover high water mark to 99%, which allows space for journals and indexes. This keeps prov_repo usage much closer to the nifi.provenance.repository.max.storage.size. +1 will merge. ---
[GitHub] nifi issue #2487: NIFI-4774: Allow user to choose which write-ahead log impl...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2487 I tested this and I was able to switch back and forth between MinimalLockingWriteAheadLog and SequentialAccessWriteAheadLog. +1 from me. It's best to make this switch while there are 0 flowfiles in the repository. With flowfiles in the system, going from MinimalLockingWAL to SequantialAccessWAL worked, but the opposite had some issues. ---
[GitHub] nifi issue #2497: NIFI-3599 Allowed back pressure object count and data size...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2497 @markap14 and @mcgilman I did consider that backpressure settings didn't really belong in AboutDTO. The BannerDTO also pulls information from nifi.properties, but I didn't think backpressure fit there either. I didn't want to further expand the API by adding something like BackpressureDTO at a /nifi-api/flow/backpressure endpoint, but maybe that's the preferred approach? Or perhaps a PropertiesDTO at /nifi-api/flow/properties to do something more generic? ---
[GitHub] nifi issue #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2458 @markap14 I added a commit to do character set validation in a property validator instead of OnScheduled. ---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request: https://github.com/apache/nifi/pull/2458#discussion_r171354166 --- Diff: nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java --- @@ -136,9 +155,16 @@ relationships = Collections.unmodifiableSet(_relationships); } +@OnScheduled --- End diff -- Very good point, and I like the customValidate approach. I'll make the change. ---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r171296519
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java
---
@@ -136,9 +155,16 @@
relationships = Collections.unmodifiableSet(_relationships);
}
+@OnScheduled
--- End diff --
Thanks for looking at this @markap14. The CHARACTER_SET_VALIDATOR
essentially says a property value is valid if an EL expression returns a
String. If someone sets it to ${system.charset} and that environment variable
is set to "FOO", for example, then when ConsumeJMS receives a TextMessage it
will throw an UnsupportedCharsetException at runtime. I thought I would
include this method to give earlier warning. PublishJMS doesn't need this
check, because the charset could be set in a flowfile attribute, and
UnsupportedCharsetException would just cause the flowfile to go to 'failure'.
If you still think the method is unnecessary, though, let me know and I can
remove it.
---
[GitHub] nifi pull request #2497: NIFI-3599 Allowed back pressure object count and da...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2497 NIFI-3599 Allowed back pressure object count and data size to be configurable Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3599 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2497.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2497 commit 5ba57cb9c10008b173e5b928979c96e9ade2f0b5 Author: Mike Moser Date: 2018-02-26T19:04:04Z NIFI-3599 Allowed back pressure object count and data size to be configurable ---
[GitHub] nifi pull request #2496: NIFIDEV-3599 Allowed back pressure object count and...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/2496 ---
[GitHub] nifi pull request #2496: NIFIDEV-3599 Allowed back pressure object count and...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2496 NIFIDEV-3599 Allowed back pressure object count and data size to be configurable Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3599 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2496.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2496 commit 0de6ff07c5739cf4e0b43917e95b9e7c35d9cb86 Author: Mike Moser Date: 2018-02-26T19:04:04Z NIFIDEV-3599 Allowed back pressure object count and data size to be configurable ---
[GitHub] nifi issue #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2458 Added a third commit to improve documentation. ---
[GitHub] nifi issue #2469: NIFI-4870 Upgraded activemq-client and activemq-broker ver...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2469 Hi @alopresto, should we update the copyright year to 2018 in the applicable NOTICE files? ---
[GitHub] nifi issue #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2458 I added a second commit, to make the character set encoding configurable for JMS TextMessages, in both PublishJMS and ConsumeJMS. Thanks @markap14 for pointing this out, because this is a good improvement. ---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167258538
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
And now that I look closer, I see there is already a precedent for this in
the code, such as in LogAttribute.
---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167255119
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
Oh, I didn't think about that approach! I like it. I'll make the changes.
---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167244022
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
I did think about this, and I chose this approach on purpose. It's kind of
the whole point of setting a default Charset in your JVM, to produce results
that are specific to your desired environment. Is this not right?
---
[GitHub] nifi issue #1240: NIFI-3039 Provenance Repository - Fix PurgeOldEvent and Ro...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1240 I think this PR is still valid, though perhaps with some tweaks to PURGE_OLD_EVENTS_HIGH_WATER, PURGE_OLD_EVENTS_LOW_WATER and ROLLOVER_HIGH_WATER as discussed in the JIRA. ---
[GitHub] nifi issue #1241: NIFI-3039 Provenance Repository - Fix PurgeOldEvent and Ro...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1241 @jskora I think you are the only one who can close this PR. Would you mind? Thanks! ---
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2458 NIFI-2630 Allow PublishJMS to send TextMessages Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-2630 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2458.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2458 commit 441784ef8a7a3b9354cd474de495ffdbfd75ee67 Author: Mike Moser Date: 2018-02-07T16:00:57Z NIFI-2630 Allow PublishJMS to send TextMessages ---
[GitHub] nifi issue #2456: Fix for unit tests that are causing build failures in cert...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2456 Nevermind my last comment. It looks like this is going to resolve both unit test failures. ---
[GitHub] nifi issue #2456: Fix for unit tests that are causing build failures in cert...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2456 @markap14 I tried your change locally and it does fix one of the two test failures. The validateConsumeWithCustomHeadersAndProperties() is fixed but validateFailOnUnsupportedMessageType() still fails. I haven't been able to figure out why, yet. ---
[GitHub] nifi issue #2445: NIFI-4834: Updated AbstractJMSProcessor to use a separate ...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2445 I get the exact same unit test failures on Ubuntu 16.04. I was working on NIFI-2630, so I thought it was my code changes, but the test failure happens when I build master without any changes. Interestingly, when I build master on Windows 10, these unit tests pass. ---
[GitHub] nifi issue #2434: NIFI-4700: Moved all PostHTTP http clients, http responses...
Github user mosermw commented on the issue:
https://github.com/apache/nifi/pull/2434
Hi @m-hogue I got an error when trying to test PostHTTP sending to a
ListenHTTP with this PR.
2018-02-02 22:15:43,007 ERROR [Timer-Driven Process Thread-7]
o.a.nifi.processors.standard.PostHTTP
PostHTTP[id=01611004-47b6-1aa6-0836-08b0dc270fae] Failed to Post
StandardFlowFileRecord[uuid=ac9ba210-264f-43
b3-bfcf-34bc4c566932,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1517609573204-1, container=default,
section=1], offset=364938,
length=121646],offset=0,name=nifi-app.log.txt,size=121646]
to http://localhost:/contentListener: sent content and received status
code 303:See Other but no Hold URI
2018-02-02 22:15:43,020 ERROR [Timer-Driven Process Thread-7]
o.a.nifi.processors.standard.PostHTTP
PostHTTP[id=01611004-47b6-1aa6-0836-08b0dc270fae]
PostHTTP[id=01611004-47b6-1aa6-0836-08b0dc270fae] failed to
process due to java.lang.IllegalStateException: Connection pool shut down;
rolling back session: {}
java.lang.IllegalStateException: Connection pool shut down
at org.apache.http.util.Asserts.check(Asserts.java:34)
at
org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:257)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at
org.apache.nifi.processors.standard.PostHTTP.onTrigger(PostHTTP.java:643)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
---
[GitHub] nifi pull request #2412: NIFI-4699 Use a filter in PostHTTP to pull flowfile...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2412 NIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose… … URL is the same Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-4699 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2412.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2412 commit bd63b11c2cda765342f747785910f04083ab3620 Author: Mike Moser Date: 2018-01-09T15:56:54Z NIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose URL is the same ---
[GitHub] nifi pull request #2381: NIFI-4697 clarified some PostHTTP documentation
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2381 NIFI-4697 clarified some PostHTTP documentation Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [N/A] Have you written or updated unit tests to verify your changes? - [N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [N/A] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [N/A] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [N/A] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-4697 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2381.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2381 commit 3f503b890aa0278b42a9d3e43c75a37dab430e38 Author: Mike Moser Date: 2018-01-05T21:13:05Z NIFI-4697 clarified some PostHTTP documentation ---
[GitHub] nifi issue #2284: NIFI-4504, NIFI-4505 added methods to MapCache API …
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2284 I rebased and force pushed the discussed changes. It appears that the Travis failure is unrelated? ---
[GitHub] nifi issue #2284: NIFI-4504, NIFI-4505 added methods to MapCache API …
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2284 Thank you Joe and Koji for taking a look at this. I struggled with deciding whether to provide a default UnsupportedOperationException or to provide the new methods in the various DistributedMapCacheClient implementations. I will happily change this PR to go with the UnsupportedOperationException route. Give me a little time to make the changes and force push, but it should be a much smaller PR and easier to review. I agree that a keySet method could become problematic in a distributed Map environment. However I believe it's important to provide as much of the Java Map interface in the distributed Map as we can, and let the implementer beware of the potential problems. I do know some folks who have extended DistributedMapCacheClient with their own implementations, and I would like the keySet method to exist to support them. ---
[GitHub] nifi issue #1637: NIFI-3648 removed cluster message copying when not in debu...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1637 Bump. I had forgotten about this, but it looks like it can still merge cleanly. ---
[GitHub] nifi issue #222: NIFI-614 Added initial support for new style JMS
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/222 Since TIBCO libraries don't fall under a license that is compatible with the Apache License 2.0, Apache NiFi cannot distribute a solution that is specific to TIBCO. But you should be able to build an extension to Apache NiFi that works for you. Here is an example for how you might approach building a NiFi extension for this [mosermw/nifi-jms-providers-bundle](https://github.com/mosermw/nifi-jms-providers-bundle). ---
[GitHub] nifi pull request #2284: NIFI-4504, NIFI-4505 added methods to MapCache API ...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2284 NIFI-4504, NIFI-4505 added methods to MapCache API … … including keySet, removeAndGet, removeByPatternAndGet cleaned up some warnings on deprecated nifi.stream.io classes I attempted to update the various implementations of DistributedMapCacheClient, but if the reviewer feels I should leave them unsupported then let me know. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-4505 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2284.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2284 commit 269fb770d793962a4dbb85b00f66b2d247ebd0fe Author: Mike Moser Date: 2017-11-15T21:54:46Z NIFI-4504, NIFI-4505 added removeAndGet, removeByPatternAndGet, and keySet methods to MapCache API cleaned up some warnings on deprecated nifi.stream.io classes ---
[GitHub] nifi issue #2260: NIFI-4589: Allow multiple keys in FetchDistributedMapCache...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2260 Code looks good, full build with contrib-check passes, and I tested a small flow with success & failure results as expected. +1 merged. There is a comment in the ProtocolHandshake class that I would modify to reflect version 3 of DistributedMapCache, but it's just a comment and I can take care of that in my upcoming PR. ---
[GitHub] nifi issue #2260: NIFI-4589: Allow multiple keys in FetchDistributedMapCache...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2260 Reviewing. I will have a PR soon against this same code, so I would rather get this one committed and rebase before I submit my PR. ---
[GitHub] nifi pull request #2041: NIFI-3736 modify default nifi.content.claim.max.app...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/2041 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2041: NIFI-3736 modify default nifi.content.claim.max.app...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/2041 NIFI-3736 modify default nifi.content.claim.max.appendable.size in nifi.properties to 1 MB You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-3736 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2041.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2041 commit 641fb4fbc2da1942fead30bf74a3d960769a12ff Author: Mike Moser Date: 2017-07-27T20:42:09Z NIFI-3736 modify default nifi.content.claim.max.appendable.size in nifi.properties to 1 MB --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2010: NIFI-3736: change to honor nifi.content.claim.max.a...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2010#discussion_r128633148
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
---
@@ -149,6 +154,10 @@ public FileSystemRepository(final NiFiProperties
nifiProperties) throws IOExcept
for (final Path path : fileRespositoryPaths.values()) {
Files.createDirectories(path);
}
+this.maxFlowFilesPerClaim =
nifiProperties.getMaxFlowFilesPerClaim();
+this.writableClaimQueue = new
LinkedBlockingQueue<>(maxFlowFilesPerClaim);
+final String maxAppendableClaimSize =
nifiProperties.getMaxAppendableClaimSize();
+this.maxAppendableClaimLength =
DataUnit.parseDataSize(maxAppendableClaimSize, DataUnit.B).intValue();
--- End diff --
In my opinion, I don't see anything wrong with a static cap, as long as
it's reasonably big enough (the 100 MB choice).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1832: NIFI-1452 on timer-driven yield, use the greater of...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1832 NIFI-1452 on timer-driven yield, use the greater of yield duration or run schedule Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-1452 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1832.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1832 commit 7d376af5020226da8e036c602fb90cfeb0530714 Author: Mike Moser Date: 2017-05-19T19:48:10Z NIFI-1452 on timer-driven yield, use the greater of yield duration or run schedule --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1732: NIFI-2752 Correct ReplaceText default pattern and unit tes...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1732 +1 contrib-check, verified unit tests before and after the fix, ran a simple flow, and looks like it matches master branch. Thanks @jskora will merge to 0.x branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1732: NIFI-2752 Correct ReplaceText default pattern and unit tes...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1732 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1723: NIFI-3223 added support for expression language to ...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1723 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1723: NIFI-3223 added support for expression language to Publish...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1723 Thanks @trkurc, with your +1 I'll go ahead and merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1723: NIFI-3223 added support for expression language to ...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1723 NIFI-3223 added support for expression language to PublishAMQP properties supporting EL are EXCHANGE and ROUTING_KEY. This is a PR for the 0.x branch, using changes from #1449. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3223 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1723.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1723 commit 27850470833e69e5978e31d8757a8940bbdc8ddb Author: Oleg Zhurakousky Date: 2017-01-27T18:41:54Z NIFI-3223 added support for expression language - EXCHANGE - ROUTING_KEY --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1699: NIFI-3725 - validate processors only when they are ...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1699 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1688: NIFI-3553 FetchFileTransfer fix
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1688 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1667: NIFI-3019 cherry-picked from NIFI-3692 into 0.x bra...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1667 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1666: NIFI-3018 removed flume-twitter-source due to usage...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1666 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1699: NIFI-3725 - validate processors only when they are ...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1699 NIFI-3725 - validate processors only when they are in STOPPED state - report validation errors via REST API on processors/services/tasks/ports only when they are in the STOPPED state Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3725 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1699.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1699 commit 4e4ffb95e260821adfca98ba03cb751bcc8c240c Author: Mike Moser Date: 2017-04-26T18:00:32Z NIFI-3725 - validate processors only when they are in STOPPED state - report validation errors via REST API on processors/services/tasks/ports only when they are in the STOPPED state --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1688: NIFI-3553 FetchFileTransfer fix
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1688 NIFI-3553 FetchFileTransfer fix This is a port from master to the 0.x branch. Reviewer can squash to remove my commit for Java 7 compatibility. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3553 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1688 commit fa012fed04a8149edf775514a9a3dc395a3f619a Author: Mark Payne Date: 2017-03-03T18:18:41Z NIFI-3553: If IOException is thrown when completing FTP transfer, do not update the FlowFile to point to the content that was streamed but instead leave FlowFile pointing to original version commit 7b4287eda1ed46e70c10d1835fdb9bb66af8b7b6 Author: Mike Moser Date: 2017-04-21T19:00:53Z NIFI-3553 added final keyword for java 7 compatibility --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1680: NIFI-3710: Port S3 Signer V4 Fixes to 0.x
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1680 Builds and executes as expected, and resolves the issue. Thanks @jvwing. +1 will merge to 0.x --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1680: NIFI-3710: Port S3 Signer V4 Fixes to 0.x
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1680 reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1670: NIFI-3702 AWS SDK Update:
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1670 Would you close this PR @jvwing? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1670: NIFI-3702 AWS SDK Update:
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1670 +1 will merge. Thanks @jvwing. It looks like I should also migrate the changes from NIFI-2763 and NIFI-2902 (V4 Signer support) over to the 0.x branch as well, to avoid breaking that functionality. I will create a new JIRA for that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1670: NIFI-3702 AWS SDK Update:
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1670 Thanks for this @jvwing. Would you mind updating the nifi-nar-bundles/nifi-aws-bundle/nifi-aws-nar/src/main/resources/META-INF/NOTICE and nifi-assembly/NOTICE files to remove the 1 line from each that says "JSON parsing and utility functions from JSON.org - Copyright 2002 JSON.org" in the Amazon Web Services SDK section? If you could take care of that in this PR, then I won't need another! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1667: NIFI-3019 cherry-picked from NIFI-3692 into 0.x bra...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1667 NIFI-3019 cherry-picked from NIFI-3692 into 0.x branch NIFI-3692 Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3019 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1667.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1667 commit b4669322abe9a9a13d951b7158577ac4fb874027 Author: Joey Frazee Date: 2017-04-11T15:19:44Z NIFI-3019 cherry-picked into 0.x branch by mose...@apache.org NIFI-3692 Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1666: NIFI-3018 removed flume-twitter-source due to usage...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1666 NIFI-3018 removed flume-twitter-source due to usage of org.json cat-x lib This change is essentially copied from master to 0.x branch Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3018 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1666.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1666 commit 9cf47f52c91eddb8ea513defd5a4a76e75ced099 Author: Mike Moser Date: 2017-04-12T15:35:19Z NIFI-3018 removed flume-twitter-source due to usage of org.json cat-x lib --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1637: NIFI-3648 removed cluster message copying when not ...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1637#discussion_r109018746
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster-protocol/src/main/java/org/apache/nifi/cluster/protocol/impl/SocketProtocolListener.java
---
@@ -134,15 +132,21 @@ public void dispatchRequest(final Socket socket) {
// unmarshall message
final ProtocolMessageUnmarshaller
unmarshaller = protocolContext.createUnmarshaller();
-final InputStream inStream = socket.getInputStream();
-final CopyingInputStream copyingInputStream = new
CopyingInputStream(inStream, maxMsgBuffer); // don't copy more than 1 MB
+final ByteCountingInputStream countingIn = new
ByteCountingInputStream(socket.getInputStream());
+InputStream wrappedInStream = countingIn;
+if (logger.isDebugEnabled()) {
+final int maxMsgBuffer = 1024 * 1024; // don't buffer
more than 1 MB of the message
+final CopyingInputStream copyingInputStream = new
CopyingInputStream(wrappedInStream, maxMsgBuffer);
+wrappedInStream = copyingInputStream;
+}
final ProtocolMessage request;
try {
-request = unmarshaller.unmarshal(copyingInputStream);
+request = unmarshaller.unmarshal(wrappedInStream);
} finally {
-receivedMessage = copyingInputStream.getBytesRead();
if (logger.isDebugEnabled()) {
--- End diff --
Excellent! I thought of using instanceof CopyingInputStream but didn't
think it would help. The potential race condition makes it useful and
necessary. I pushed a fix, will squash if needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1637: NIFI-3648 removed cluster message copying when not ...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1637 NIFI-3648 removed cluster message copying when not in debug mode I expect NIFI-3648 could have several PRs, and that they will not be included in 1.2.0 release, so review of this can be delayed until the rest of that ticket is PRed and reviewed. I just wanted to get this in so I don't forget. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3648-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1637.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1637 commit 55d93ccbec8fd7eaede0759e7abb550c75d36a63 Author: Mike Moser Date: 2017-03-30T14:34:26Z NIFI-3648 removed message copying when not in debug mode --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1621: NIFI-3304 removed unneeded Java 8 codecache info fr...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1621 NIFI-3304 removed unneeded Java 8 codecache info from Admin Guide Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-3304 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1621.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1621 commit 858bc4dc49e299f815c5ad3ae5e6dbdcebfcd575 Author: Mike Moser Date: 2017-03-24T21:44:07Z NIFI-3304 removed unneeded Java 8 codecache info from Admin Guide --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1603: NIFI-2481 improved content repo archive description...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1603 NIFI-2481 improved content repo archive description in Admin Guide Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi nifi-2481 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1603.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1603 commit 1e83e605e00912f27551f8096ec0094b86aa159d Author: Mike Moser Date: 2017-03-17T20:53:52Z NIFI-2481 improved content repo archive description in Admin Guide --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1580: Fix for Windows FileStore issue NIFI-3579
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1580#discussion_r106473572
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
---
@@ -392,8 +399,11 @@ public long getContainerUsableSpace(String
containerName) throws IOException {
if (path == null) {
throw new IllegalArgumentException("No container exists with
name " + containerName);
}
-
-return Files.getFileStore(path).getUsableSpace();
+long usableSpace=path.toFile().getUsableSpace();
+if(usableSpace==0) {
+throw new RuntimeException("System returned usable space of
the partition for " + containerName + " is zero byte. Nifi can not create a
zero sized FileSystemRepository");
--- End diff --
Hi @PuspenduBanerjee, since getContainerCapacity() and
getContainerUsableSpace() are public methods, I would prefer that we try to
maintain their existing contract. I would prefer to avoid even an extremely
small chance of a RuntimeException.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1580: Fix for Windows FileStore issue NIFI-3579
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1580#discussion_r106435047
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
---
@@ -392,8 +399,11 @@ public long getContainerUsableSpace(String
containerName) throws IOException {
if (path == null) {
throw new IllegalArgumentException("No container exists with
name " + containerName);
}
-
-return Files.getFileStore(path).getUsableSpace();
+long usableSpace=path.toFile().getUsableSpace();
+if(usableSpace==0) {
+throw new RuntimeException("System returned usable space of
the partition for " + containerName + " is zero byte. Nifi can not create a
zero sized FileSystemRepository");
--- End diff --
After more thought, I agree with the concerns that @markap14 raised. I
think the constructor can throw RuntimeException as it exists now. But I think
the getContainerCapacity() should throw IOException and
getContainerUsableSpace() should just return the 0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1580: Fix for Windows FileStore issue NIFI-3579
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1580 +1 looks good, passes checkstyle, runs in NiFi on Windows when creating content_repository on startup and when it already exists. I will squash and merge to master. Thanks @PuspenduBanerjee --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1580: Fix for Windows FileStore issue NIFI-3579
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1580#discussion_r105747874
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java
---
@@ -199,13 +199,13 @@ public FileSystemRepository(final NiFiProperties
nifiProperties) throws IOExcept
for (final Map.Entry container :
containers.entrySet()) {
final String containerName = container.getKey();
-final long capacity =
Files.getFileStore(container.getValue()).getTotalSpace();
+final long capacity =
container.getValue().toFile().getTotalSpace();
--- End diff --
If there is a problem, FileStore.getTotalSpace() throws IOException but
File.getTotalSpace() returns 0. I think you should throw an IOException if
capacity == 0, in order to maintain similar behavior here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1548: Fixed a bug that nifi-jms-cf-service cannot have a variabl...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1548 @ShellyLC If you wouldn't mind opening a NIFI JIRA ticket for this bug, we would really appreciate it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1214: NIFI-2876 refactored demarcators into a common abst...
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1214#discussion_r102828860
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/util/AbstractDemarcator.java
---
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io.util;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+
+import org.apache.nifi.stream.io.exception.TokenTooLargeException;
+
+/**
+ * Base class for implementing streaming demarcators.
+ *
+ * NOTE: Not intended for multi-thread usage hence not Thread-safe.
+ *
+ */
+abstract class AbstractDemarcator implements Closeable {
+
+final static int INIT_BUFFER_SIZE = 8192;
+
+private final InputStream is;
+
+private final int initialBufferSize;
+
+private final int maxDataSize;
+
+byte[] buffer;
+
+int index;
+
+int mark;
+
+long offset;
+
+int bufferLength;
+
+/**
+ * Constructs an instance of demarcator with provided {@link
InputStream}
+ * and max buffer size. Each demarcated token must fit within max
buffer
+ * size, otherwise the exception will be raised.
+ */
+AbstractDemarcator(InputStream is, int maxDataSize) {
+this(is, maxDataSize, INIT_BUFFER_SIZE);
+}
+
+/**
+ * Constructs an instance of demarcator with provided {@link
InputStream}
+ * and max buffer size and initial buffer size. Each demarcated token
must
+ * fit within max buffer size, otherwise the exception will be raised.
+ */
+AbstractDemarcator(InputStream is, int maxDataSize, int
initialBufferSize) {
+this.validate(is, maxDataSize, initialBufferSize);
+this.is = is;
+this.initialBufferSize = initialBufferSize;
+this.buffer = new byte[initialBufferSize];
+this.maxDataSize = maxDataSize;
+}
+
+@Override
+public void close() throws IOException {
+// noop
+}
+
+/**
+ * Will fill the current buffer from current 'index' position,
expanding it
+ * and or shuffling it if necessary. If buffer exceeds max buffer size
a
+ * {@link TokenTooLargeException} will be thrown.
+ *
+ * @throws IOException
+ * if unable to read from the stream
+ */
+void fill() throws IOException {
+if (this.index >= this.buffer.length) {
+if (this.mark == 0) { // expand
+byte[] newBuff = new byte[this.buffer.length +
this.initialBufferSize];
+System.arraycopy(this.buffer, 0, newBuff, 0,
this.buffer.length);
+this.buffer = newBuff;
+} else { // shuffle
+int length = this.index - this.mark;
+System.arraycopy(this.buffer, this.mark, this.buffer, 0,
length);
+this.index = length;
+this.mark = 0;
+}
+}
+
+int bytesRead;
+do {
+bytesRead = this.is.read(this.buffer, this.index,
this.buffer.length - this.index);
--- End diff --
Generic InputStreams that are backed by a network socket implementation
actually can return 0 temporarily, before continuing to return good data. Not
sure if this class will ever be used in that situation, though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1509: NIFI-3362 update FlowConfiguration.xsd to allow all curren...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1509 Thank you @apiri ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1509: NIFI-3362 update FlowConfiguration.xsd to allow all...
Github user mosermw closed the pull request at: https://github.com/apache/nifi/pull/1509 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1509: NIFI-3362 update FlowConfiguration.xsd to allow all...
GitHub user mosermw opened a pull request: https://github.com/apache/nifi/pull/1509 NIFI-3362 update FlowConfiguration.xsd to allow all current time period units @apiri I pulled out the new unit test that was in master from this 0.x PR. I would have needed a new flow.xml for the unit test to validate. I didn't feel it was critical to include the test in 0.x considering that we have it in master, moving forward. Please let me know if you disagree, and I will amend this PR. Thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/mosermw/nifi NIFI-3362-0x Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1509.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1509 commit a51047a3b82a552bd4c341954466d742697612e0 Author: Mike Moser Date: 2017-02-10T20:06:16Z NIFI-3362 update FlowConfiguration.xsd to allow all current time period units --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1481: NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatExce...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1481 @trixpan the "This closes" commit hook doesn't work on the 0.x branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1481: NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatExce...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/1481 Reviewed, along with #1475. Licensing concern about the code looks to be resolved. +1 will merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
