[GitHub] nifi pull request #3053: NIFI-5660: JMSPublisher should not set header prope...
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3053#discussion_r223751456
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java
---
@@ -89,11 +89,14 @@ void setMessageHeaderAndProperties(final Message
message, final Map entry :
flowFileAttributesToSend.entrySet()) {
try {
if (entry.getKey().equals(JmsHeaders.DELIVERY_MODE)) {
-
message.setJMSDeliveryMode(Integer.parseInt(entry.getValue()));
+this.jmsTemplate.setExplicitQosEnabled(true);
--- End diff --
No, it should not. The only changes is setting these values through the
JmsTemplate vice the Message itself due to stated limitation of the API. The
net effect is the same.
---
[GitHub] nifi pull request #3053: NIFI-5660: JMSPublisher should not set header prope...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/3053 NIFI-5660: JMSPublisher should not set header properties directly in ⦠â¦the message Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. Integration tests validated expected behavior. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5660 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3053.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3053 commit b6e5ca9d44e54fd3fc2cdc40b4a21a1b6783a452 Author: Mark Bean Date: 2018-10-05T14:08:28Z NIFI-5660: JMSPublisher should not set header properties directly in the message ---
[GitHub] nifi issue #2957: NIFI-5543: fix connection path is not visualizing as selec...
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2957 @scottyaslan This behavior has been confirmed on Firefox 56.0 and Chrome 50.0. Granted these are not Current or (Current -1). What browser/version are you using? ---
[GitHub] nifi issue #2957: NIFI-5543: fix connection path is not visualizing as selec...
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2957 @scottyaslan The connections do not need to be overlapping to experience the bad behavior. As the JIRA ticket says, "Select a connection. Then, select another connection on its bend point." These connections do not need to be overlapping. In both cases, the second connection can be manipulated while the first connection still displays as highlighted. That is the behavior prior to this fix, that is. ---
[GitHub] nifi pull request #2957: NIFI-5543: fix connection path is not visualizing a...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2957 NIFI-5543: fix connection path is not visualizing as selected when hi⦠â¦dden linepoint element is clicked Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. Fix was verified by installing and manual UI manipulation. Now, when a bend point of a currently unselected connection is clicked, that connection becomes selected/highlighted. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5543 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2957.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2957 commit 4b719c6d9019e1aeabfd5205540ede5653d6236c Author: Mark Bean Date: 2018-08-21T15:37:25Z NIFI-5543: fix connection path is not visualizing as selected when hidden linepoint element is clicked ---
[GitHub] nifi-site pull request #27: Added GPG Guide
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi-site/pull/27#discussion_r202515416 --- Diff: src/pages/markdown/gpg.md --- @@ -0,0 +1,522 @@ +--- +title: GPG (Gnu Privacy Guard / Open PGP) Signatures +--- + +# Apache NiFi Release Signatures and Code Signing + +The purpose of this document is to capture and describe the steps involved in generating and verifying cryptographic signatures of official releases of Apache NiFi, as well as configuring cryptographic signatures of individual code commits. It is written for contributors, committers, and users of Apache NiFi (and related applications). + +## Table of Contents + * [The Objective](#the-objective) + * [Background Material](#background-material) + * [Terms](#terms) + * [Variable Reference Substitutions](#variable-reference-substitutions) + * [Download GPG](#download-gpg) + * [Setting up your key](#setting-up-your-key) + * [Publish your key](#publish-your-key) + * [Import another GPG key](#import-another-gpg-key) + * [Verify a key](#verify-a-key) + * [Web of trust](#web-of-trust) + * [Set up Git with your key](#set-up-git-with-your-key) + * [Sign commits](#sign-commits) + * [Verifying a signature](#verifying-a-signature) + * [Set up GitHub with your key](#set-up-github-with-your-key) + * [Signing a release artifact](#signing-a-release-artifact) + * [Verifying a release signature](#verifying-a-release-signature) + +## The Objective + +Our aim is to instruct users on how to sign their commits, verify other's signatures, and do the same for official releases of Apache NiFi. + +## Background Material + + - These documents are helpful for general environmental setup to perform GPG signing and signature verification: +- [Apache PGP Info][apache-pgp] +- [Apache Release Signing][apache-release-signing] +- [Git Ready: Signing Releases with GPG][git-sign-tag-instructs] +- [RFC 4880: IETF Standard Spec for OpenPGP][rfc-4880] +- [GitHub Blog: GPG Signature Verification][github-gpg-signing] +- [Git Ready: gpg-sign releases][git-sign-tag-instructs] +- [GitHub Help: Signing Commits Using GPG][github-help-gpg] +- [GitHub Help: Telling Git About Your GPG Key][git-config-gpg] +- [Git Docs: Git Tools Signing Your Work][git-gpg] +- [PGP Web of Trust][web-of-trust] + +## Terms --- End diff -- Add definitions of "PGP" and "GPG" to terms. Useful on its own, but also these acronyms are so similar and can look like an inadvertent typo. ---
[GitHub] nifi pull request #2873: NIFI-5368 controller services validated prior to en...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2873 NIFI-5368 controller services validated prior to enabling; referenced⦠⦠controller services must be enabled for referencing component to be valid (mock framework) Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5368 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2873.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2873 commit 3a5d94457b9f470a0b620364f11ceb01eeb7a6ee Author: Mark Bean Date: 2018-07-05T19:35:15Z NIFI-5368 controller services validated prior to enabling; referenced controller services must be enabled for referencing component to be valid (mock framework) ---
[GitHub] nifi pull request #2847: NIFI-5377 prevent infinite loop if a controller ser...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2847 NIFI-5377 prevent infinite loop if a controller service circular refe⦠â¦rence exists Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5377 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2847.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2847 commit dc43aa9d956826a777ae2a0a647356ed7f28a9ba Author: Mark Bean Date: 2018-07-06T12:29:00Z NIFI-5377 prevent infinite loop if a controller service circular reference exists ---
[GitHub] nifi issue #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2703 I like the proposed changes. It makes the authorization process a bit cleaner. +1 ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r194730011
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1338,6 +1339,67 @@ private void authorizeReplay(final
ProvenanceEventRecord event) {
dataAuthorizable.authorize(authorizer, RequestAction.WRITE, user,
eventAttributes);
}
+private AuthorizationResult
checkAuthorizationForData(ProvenanceEventRecord event) {
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+final Authorizable dataAuthorizable;
+if (event.isRemotePortType()) {
+dataAuthorizable =
flowController.createRemoteDataAuthorizable(event.getComponentId());
+} else {
+dataAuthorizable =
flowController.createLocalDataAuthorizable(event.getComponentId());
+}
+
+final Map eventAttributes = event.getAttributes();
+
+// ensure we can read the data
+return dataAuthorizable.checkAuthorization(authorizer,
RequestAction.READ, user, eventAttributes);
+}
+
+private AuthorizationResult checkAuthorizationForProvenanceData(final
ProvenanceEventRecord event) {
+final ProcessGroup rootGroup =
flowController.getGroup(getRootGroupId());
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+final String componentId = event.getComponentId();
+Connectable connectable;
+String targetId = null;
+// check if the component is the rootGroup
+if (getRootGroupId().equals(componentId)) {
+targetId = componentId;
+}
+if (targetId == null) {
+// check if the component is a processor
+connectable = rootGroup.findProcessor(componentId);
+if (connectable == null) {
+// if the component id is not a processor then consider a
connection
+connectable =
rootGroup.findConnection(componentId).getSource();
+
+if (connectable == null) {
+throw new ResourceNotFoundException("The component
that generated this event is no longer part of the data flow");
+}
+}
+targetId = connectable.getIdentifier();
+}
+final Authorizable provenanceDataAuthorizable =
flowController.createProvenanceDataAuthorizable(targetId);
+
+return provenanceDataAuthorizable.checkAuthorization(authorizer,
RequestAction.READ, user);
+}
+
+private AuthorizationResult checkConnectableAuthorization(final String
componentId) {
+final ProcessGroup rootGroup =
flowController.getGroup(getRootGroupId());
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+if (rootGroup.getIdentifier().equals(componentId)) {
+return rootGroup.checkAuthorization(authorizer,
RequestAction.READ, user);
+}
+Connectable connectable =
rootGroup.findLocalConnectable(componentId);
--- End diff --
Will findLocalConnectable() versus findProcessor() include connections as
well? If so, then this should return to findProcessor() to account for
connections and subsequently finding the connection's source component.
---
[GitHub] nifi issue #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2703 I believe the latest changes are demonstrating the intended functionality. Provenance events are only listed in the query results if the user has 'view provenance' on the corresponding component; flowfile content in the event details is only visible based on 'view the data' policy; component name and type is only visible based on 'view the component' policy (replaced with generic info such as UUID in place of name when policy is lacking.) I'll do some further testing today. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r194718895
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1489,6 +1492,12 @@ public int compare(AttributeDTO a1, AttributeDTO a2)
{
dto.setChildUuids(childUuids);
}
+// lineage duration
+if (event.getLineageStartDate() > 0) {
--- End diff --
lineage duration was pulled out specifically because there was a case in
which the duration was not properly populated. This was during early testing
and may now be corrected by other changes. I will try to replicate.
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r194718350
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1338,6 +1339,67 @@ private void authorizeReplay(final
ProvenanceEventRecord event) {
dataAuthorizable.authorize(authorizer, RequestAction.WRITE, user,
eventAttributes);
}
+private AuthorizationResult
checkAuthorizationForData(ProvenanceEventRecord event) {
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+final Authorizable dataAuthorizable;
+if (event.isRemotePortType()) {
+dataAuthorizable =
flowController.createRemoteDataAuthorizable(event.getComponentId());
+} else {
+dataAuthorizable =
flowController.createLocalDataAuthorizable(event.getComponentId());
+}
+
+final Map eventAttributes = event.getAttributes();
+
+// ensure we can read the data
+return dataAuthorizable.checkAuthorization(authorizer,
RequestAction.READ, user, eventAttributes);
+}
+
+private AuthorizationResult checkAuthorizationForProvenanceData(final
ProvenanceEventRecord event) {
--- End diff --
I modified this method and checkConnectableAuthorization() to accomodate a
Process Group being the event component. This is the case for DOWNLOAD
provenance events.
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r194513579
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
---
@@ -4919,6 +4925,22 @@ private void updateRemoteProcessGroups() {
return new
ArrayList<>(provenanceRepository.getEvents(firstEventId, maxRecords));
}
+public AuthorizationResult checkConnectableAuthorization(final String
componentId) {
--- End diff --
Correct. This was moved to ControllerFacade.java. I will remove it from
FlowController.java.
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r194510876
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1389,104 +1420,119 @@ private ProvenanceEventDTO
createProvenanceEventDto(final ProvenanceEventRecord
// sets the component details if it can find the component still
in the flow
setComponentDetails(dto);
-// only include all details if not summarizing
-if (!summarize) {
-// convert the attributes
-final Comparator attributeComparator = new
Comparator() {
-@Override
-public int compare(AttributeDTO a1, AttributeDTO a2) {
-return
Collator.getInstance(Locale.US).compare(a1.getName(), a2.getName());
-}
-};
+//try {
+//AuthorizationResult result =
flowController.checkConnectableAuthorization(event.getComponentId());
+AuthorizationResult result =
checkConnectableAuthorization(event.getComponentId());
+if (Result.Denied.equals(result.getResult())) {
+dto.setComponentType("Processor"); // is this always a
Processor?
+dto.setComponentName(dto.getComponentId());
+dto.setEventType("UNKNOWN");
--- End diff --
If we choose to _not_ redact event type, that makes life easier. Currently,
it displays "UNKNOWN" in the table (when 'view provenance' is enabled and 'view
the component' is not). But, the event type IS diplayed in the lineage graph.
We need to get to consistency one way or the other on this. I'm leaning towards
allowing the event type info to be visible since this is a characteristic of
provenance (i.e. 'view provenance') and not a characteristic of 'view the
component'.
---
[GitHub] nifi issue #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2703 When calling getEvent() from the provenance repository, the user is authorized for the event (including component level authorization). See ControllerFacade.java:1353. This getEvent() method call is prior to createProvenanceEventDto(). So, it would be redundant to authorize the user for the event inside createProvenanceEventDto() as any unauthorized events will have already been filtered out. The original approach was to exclude all events from a provenance query result for which the user is not authorized (e.g. the user is not in the 'view provenance' component level policy). Therefore, it should not be necessary to perform your point #2 above. For point #3 and a slight refactor of authorizeReplay(), I've renamed it to authorizeData(). And, removed the duplicate authorization block from getProvenanceEvent(). Instead, the createProvenanceEventDto() will perform the data authorization prior to the if !summarize block. In this way, the event will need to be authorized for data access as well as not summarized in order for the dto to populate the attributes and content. I also updated some authorization unit tests with more detailed expected results. And, rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r192413226
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1359,7 +1363,12 @@ public ProvenanceEventDTO getProvenanceEvent(final
Long eventId) {
} else {
dataAuthorizable =
flowController.createLocalDataAuthorizable(event.getComponentId());
}
-dataAuthorizable.authorize(authorizer, RequestAction.READ,
NiFiUserUtils.getNiFiUser(), attributes);
+// If not authorized for 'view the data', create only
summarized provenance event
--- End diff --
My only concern with the approach you outlined is the additional
authorizations calls to determine "if the user is allowed". What you suggest
requires up to 2 additional authorizations per provenance event. Already on
busy systems, we have observed authorizing the user to each provenance event as
a limiting factor (it can result in provenance becoming unusable).
Having said that, unless you think of another approach which would require
fewer authorizations calls, I'll proceed as you recommend. I suspect there may
be a future JIRA ticket to address the provenance query/authorization impact
anyhow; if so, this can be addressed at that time. We won't know for sure if
this is a problem until we get the current fix into an appropriately loaded
test environment.
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r191053294
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java
---
@@ -1359,7 +1363,12 @@ public ProvenanceEventDTO getProvenanceEvent(final
Long eventId) {
} else {
dataAuthorizable =
flowController.createLocalDataAuthorizable(event.getComponentId());
}
-dataAuthorizable.authorize(authorizer, RequestAction.READ,
NiFiUserUtils.getNiFiUser(), attributes);
+// If not authorized for 'view the data', create only
summarized provenance event
--- End diff --
The summarized event does seem to exclude other details that do not fall
under 'view the data' (i.e. attributes and content.) For example, event
duration and parent/child UUIDs. It seems either more event details besides
lineageStartDate need to be moved out of the "if (!summarized)" block, or...
what else would you suggest? A new method to generate the ProvenanceEventDTO
which explicitly excludes all attributes and content?
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2703#discussion_r191052587
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-file-authorizer/src/main/java/org/apache/nifi/authorization/RoleAccessPolicy.java
---
@@ -63,7 +63,7 @@ public String getAction() {
final Set provenancePolicies = new HashSet<>();
provenancePolicies.add(new
RoleAccessPolicy(ResourceType.Provenance.getValue(), READ_ACTION));
if (rootGroupId != null) {
-provenancePolicies.add(new
RoleAccessPolicy(ResourceType.Data.getValue() +
ResourceType.ProcessGroup.getValue() + "/" + rootGroupId, READ_ACTION));
+provenancePolicies.add(new
RoleAccessPolicy(ResourceType.ProvenanceData.getValue() +
ResourceType.ProcessGroup.getValue() + "/" + rootGroupId, READ_ACTION));
--- End diff --
Agree. Added Data back in.
---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052575 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ nifi.security.identity.mapping.value.kerb=$1@$2 -nifi.security.identity.mapping.transform.kerb=NONE The last segment of each property is an identifier used to associate the pattern with the replacement value. When a user makes a request to NiFi, their identity is checked to see if it matches each of those patterns in lexicographical order. For the first one that matches, the replacement specified in the `nifi.security.identity.mapping.value.` property is used. So a login with `CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US` matches the DN mapping pattern above and the DN mapping value `$1@$2` is applied. The user is normalized to `localhost@Apache NiFi`. -In addition to mapping a transform may be applied. The supported versions are NONE (no transform applied), LOWER (identity lowercased), and UPPER (identity uppercased). If not specified, the default value is NONE. --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052573 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ nifi.security.identity.mapping.value.kerb=$1@$2 -nifi.security.identity.mapping.transform.kerb=NONE The last segment of each property is an identifier used to associate the pattern with the replacement value. When a user makes a request to NiFi, their identity is checked to see if it matches each of those patterns in lexicographical order. For the first one that matches, the replacement specified in the `nifi.security.identity.mapping.value.` property is used. So a login with `CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US` matches the DN mapping pattern above and the DN mapping value `$1@$2` is applied. The user is normalized to `localhost@Apache NiFi`. -In addition to mapping a transform may be applied. The supported versions are NONE (no transform applied), LOWER (identity lowercased), and UPPER (identity uppercased). If not specified, the default value is NONE. --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052568 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ nifi.security.identity.mapping.value.kerb=$1@$2 -nifi.security.identity.mapping.transform.kerb=NONE --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052564 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052561 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on a diff in the pull request: https://github.com/apache/nifi/pull/2703#discussion_r191052566 --- Diff: nifi-docs/src/main/asciidoc/administration-guide.adoc --- @@ -3424,27 +3429,13 @@ The following examples demonstrate normalizing DNs from certificates and princip nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 -nifi.security.identity.mapping.transform.dn=NONE nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ nifi.security.identity.mapping.value.kerb=$1@$2 -nifi.security.identity.mapping.transform.kerb=NONE --- End diff -- Somehow, there was a bad rebase to master which removed some recently modified lines. Re-rebased to master. ---
[GitHub] nifi issue #2703: NIFI-4907: add 'view provenance' component policy
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2703 Fixed conflicts with master. Added NIFI-5207 since it is from the same policy refactor and required minimal code change. ---
[GitHub] nifi pull request #2703: NIFI-4907: add 'view provenance' component policy
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2703 NIFI-4907: add 'view provenance' component policy Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-4907 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2703.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2703 commit eed76e9aa2891eeccb7ef4bb4a5890a178aa7a8f Author: Mark Bean Date: 2018-05-12T20:32:29Z NIFI-4907: add 'view provenance' component policy ---
[GitHub] nifi pull request #2694: NIFI-5180: update JMS additional details to set Des...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2694 NIFI-5180: update JMS additional details to set Destination Type to R⦠â¦equired, default 'QUEUE' Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5180 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2694.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2694 commit d7a705ccad73d2305768dafcd186c248b7ab69b9 Author: Mark Bean Date: 2018-05-10T00:24:19Z NIFI-5180: update JMS additional details to set Destination Type to Required, default 'QUEUE' ---
[GitHub] nifi pull request #2634: NIFI-5075: Do not execute Funnels with no outgoing ...
Github user markobean commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2634#discussion_r181735366
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/tasks/ConnectableTask.java
---
@@ -103,8 +104,24 @@ private boolean isYielded() {
return connectable.getYieldExpiration() >
System.currentTimeMillis();
}
+/**
+ * Make sure processor has work to do. This means that it meets one of
these criteria:
+ *
+ * It is annotated with @TriggerWhenEmpty
+ * It has no incoming connections
+ * All incoming connections are self-loops
+ * It has data in an incoming Connection
+ * AND It is not a Funnel without outgoing connections
+ *
+ *
+ * @return true if there is work to do, otherwise false
+ */
private boolean isWorkToDo() {
-return connectable.isTriggerWhenEmpty() ||
!connectable.hasIncomingConnection() || !hasNonLoopConnection ||
Connectables.flowFilesQueued(connectable);
+return connectable.isTriggerWhenEmpty()
+|| !connectable.hasIncomingConnection()
--- End diff --
This logic seems inverted unnecessarily. Why is
'connectable.hasIncomingConnection()' negated? This leads to the method
returning true when there are no incoming connections. Similarly, is it the
intent to indicate no work to do if the only incoming connections are looped
connections?
---
[GitHub] nifi pull request #2617: NIFI-5055 added ability to unpenalize MockFlowFile ...
Github user markobean closed the pull request at: https://github.com/apache/nifi/pull/2617 ---
[GitHub] nifi issue #2617: NIFI-5055 added ability to unpenalize MockFlowFile directl...
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2617 Changes committed to master. Closing PR. ---
[GitHub] nifi issue #2617: NIFI-5055 added ability to unpenalize MockFlowFile directl...
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2617 So, does this PR need to be manually closed with the "Close and comment" button? ---
[GitHub] nifi issue #2617: NIFI-5055 added ability to unpenalize MockFlowFile directl...
Github user markobean commented on the issue: https://github.com/apache/nifi/pull/2617 Updated to use assertTrue and assertFalse. I had simply borrowed assertEquals from the previous unit test. So, while updating, I updated that test to use assertTrue as well. Commit, squashed and pushed. ---
[GitHub] nifi pull request #2617: NIFI-5055 added ability to unpenalize MockFlowFile ...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2617 NIFI-5055 added ability to unpenalize MockFlowFile directly or from M⦠â¦ockProcessSession Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-5055 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2617.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2617 commit 0e2cc73c4aac0a04a84ad921fed0ce0b0c7cd2a6 Author: Mark Bean Date: 2018-04-08T19:03:09Z NIFI-5055 added ability to unpenalize MockFlowFile directly or from MockProcessSession ---
[GitHub] nifi pull request #2559: NIFI-4658 set Maximum Number of Entries to required...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2559 NIFI-4658 set Maximum Number of Entries to required with default valu⦠â¦e of Integer.MAX_VALUE. Fast fail of FlowFiles having fragment.count greater than Max Entries. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-4658 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2559.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2559 commit c82b22db28b59ef6e03ec3b3bb47eeae5ee14daa Author: Mark Bean Date: 2018-03-16T23:43:29Z NIFI-4658 set Maximum Number of Entries to required with default value of Integer.MAX_VALUE. Fast fail of FlowFiles having fragment.count greater than Max Entries. ---
[GitHub] nifi pull request #2557: NIFI-4950 Defining behavior for MergeContent when m...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2557 NIFI-4950 Defining behavior for MergeContent when more than 1 FlowFil⦠â¦e has the same fragment.index value Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-4950 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2557.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2557 commit 39a4a71a22c80f2a9fff94f46e0ea3c9b3716836 Author: Mark Bean Date: 2018-03-15T23:06:00Z NIFI-4950 Defining behavior for MergeContent when more than 1 FlowFile has the same fragment.index value ---
[GitHub] nifi pull request #2258: NIFI-4072 replace keyword RELEASE with actual versi...
GitHub user markobean opened a pull request: https://github.com/apache/nifi/pull/2258 NIFI-4072 replace keyword RELEASE with actual version number Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? single ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? dependency license is MIT. And this dependency is not new, only version is specified explicitly now. - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markobean/nifi NIFI-4072 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2258.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2258 commit 34fa64ad03d7d3d468c51cc7e5adf735e8a8c390 Author: Mark Bean Date: 2017-11-09T01:02:03Z NIFI-4072 replace keyword RELEASE with actual version number ---
