date:20161119

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

2016-11-19 Thread alopresto

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1247#discussion_r88792883
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/service/StandardControllerServiceNode.java
 ---
@@ -121,6 +122,11 @@ public Resource getResource() {
 }
 
 @Override
+public boolean isRestricted() {
--- End diff --

Could this method be moved to `AbstractConfiguredComponent`? It seems like 
each place where it is implemented independently also extends that abstract 
class. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

2016-11-19 Thread alopresto

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1247#discussion_r88792735
  
--- Diff: nifi-docs/src/main/asciidoc/developer-guide.adoc ---
@@ -565,6 +565,23 @@ for instance, they should not be
 relied upon for critical business logic.
 
 
+[[restricted]]
+=== Restricted
+
+A Restricted component is one that can be used to execute arbitrary 
unsanitized code provided by the operator
+through the NiFi REST API/UI or can be used to obtain or alter data on the 
NiFi host system using the NiFi OS
+credentials. These components could be used by an otherwise authorized 
NiFi user to go beyond the intended use of
+the application, escalate privilege, or could expose data about the 
internals of the NiFi process or the host
+system. All of these capabilities should be considered privileged, and 
admins should be aware of these
+capabilities and explicitly enable them for a subset of trusted users.
+
+A Processor, Controller Service, or Reporting Task can be marked with the 
@Restricted annotation. This
--- End diff --

Maybe indicate `@Restricted` in code-formatting for consistency throughout 
document?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3050) Restrict dangerous processors to special permission

2016-11-19 Thread Andy LoPresto (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680338#comment-15680338
 ] 

Andy LoPresto commented on NIFI-3050:
-

I commented on [PR 1247|https://github.com/apache/nifi/pull/1247] with my 
feedback on the proposed solution. I really like it and had some minor 
suggestions to improve user experience, including:
* increased visibility of restricted icon/textual indicators on existing 
components on canvas
* searchability of restricted processors in Add Processor dialog
* increased visibility of restricted information in component documentation 
(currently, only present at the bottom of the documentation page). I think a 
large icon or other indicator at the top of the documentation, which could link 
to the additional description or explanation of risk at the bottom, would be 
valuable
* an auto-generated canonical list of restricted processors, controller 
services, and reporting tasks should be created at build time and populated 
into the documentation (and available on the main documentation site) to be 
quickly enumerated

I did positively verify the behavior of the proposed solution and just wanted 
to bring the discussion of these enhancements back to the main conversation to 
get commentary. I noted in the PR review that I am ok merging the current work 
as it satisfies the Jira and filing these enhancements separately as a minor 
issue if necessary for time constraints on the vote. 

> Restrict dangerous processors to special permission
> ---
>
> Key: NIFI-3050
> URL: https://issues.apache.org/jira/browse/NIFI-3050
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>Assignee: Matt Gilman
>Priority: Blocker
>  Labels: security
> Fix For: 1.1.0
>
>
> As evidenced by [NIFI-3045] and other discoveries (e.g. using an 
> {{ExecuteScript}} processor to iterate over a {{NiFiProperties}} instance 
> after the application has already decrypted the sensitive properties from the 
> {{nifi.properties}} file on disk, using a {{GetFile}} processor to retrieve 
> {{/etc/passwd}}, etc.) NiFi is a powerful tool which can allow unauthorized 
> users to perform malicious actions. While no tool as versatile as NiFi will 
> ever be completely immune to insider threat, to further restrict the 
> potential for abuse, certain processors should be designated as 
> {{restricted}}, and these processors can only be added to the canvas or 
> modified by users who, along with the proper permission to modify the canvas, 
> have a special permission to interact with these "dangerous" processors. 
> From the [Security Feature 
> Roadmap|https://cwiki.apache.org/confluence/display/NIFI/Security+Feature+Roadmap]:
> {quote}
> Dangerous Processors
> * Processors which can directly affect behavior/configuration of NiFi/other 
> services
> - {{GetFile}}
> - {{PutFile}}
> - {{ListFile}}
> - {{FetchFile}}
> - {{ExecuteScript}}
> - {{InvokeScriptedProcessor}}
> - {{ExecuteProcess}}
> - {{ExecuteStreamCommand}}
> * These processors should only be creatable/editable by users with special 
> access control policy
> * Marked by {{@Restricted}} annotation on processor class
> * All flowfiles originating/passing through these processors have special 
> attribute/protection
> * Perhaps *File processors can access a certain location by default but 
> cannot access the root filesystem without special user permission?
> {quote}
> [~mcgilman] and I should have a PR for this tomorrow. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

2016-11-19 Thread alopresto

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1247#discussion_r88792639
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/integration/accesscontrol/ITProcessorAccessControl.java
 ---
@@ -399,6 +400,43 @@ public void testNoneUserDeleteProcessor() throws 
Exception {
 verifyDelete(helper.getNoneUser(), NONE_CLIENT_ID, 403);
 }
 
+/**
+ * Tests attempt to create a restricted processor.
+ *
+ * @throws Exception
--- End diff --

Checkstyle fails on Javadoc with empty message. I changed the line to `* 
@throws Exception if there is an error creating this processor` to resolve. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1247: Introducing restricted components which require additional...

2016-11-19 Thread alopresto

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/1247

The issues with cleaning the content_repository were local to my machine. I
resolved those with @markap14 's help. Now I set up two client certificates and
two user identities -- myself and "Matt". We have the same permissions except
that Andy can add restricted components and Matt cannot.
![Different access policies for restricted
components](https://cloud.githubusercontent.com/assets/798465/20460012/74e3d204-ae8a-11e6-9f19-2d838d306815.png)

I then began adding components while logged in as Andy and Matt in separate
windows. Andy was able to add and modify *restricted* components, while Matt
was not.
https://cloud.githubusercontent.com/assets/798465/20460020/ac5564c8-ae8a-11e6-94c0-52e278122c15.png;>

![Matt cannot add a restricted
component](https://cloud.githubusercontent.com/assets/798465/20460024/b81ade00-ae8a-11e6-859f-0fa338ebd8a2.png)

![Matt cannot modify an existing restricted
component](https://cloud.githubusercontent.com/assets/798465/20460031/d75ab3b2-ae8a-11e6-8e5e-301ff0f3b1af.png)

I really like the small shield icon in the Add Processor dialog which has
hover text explaining the restriction.
![Hover text explaining the
restriction](https://cloud.githubusercontent.com/assets/798465/20460037/fe5d8dc2-ae8a-11e6-95df-6b23e9799115.png)

I think we can do more to indicate the restrictions, however. In the Add
Processor dialog, I think it makes sense to put a textual indicator in the
description area at the bottom above the processor description -- perhaps red
text saying "Restricted Processor". I also think we should update the
restricted processors with a *Restricted* tag so they can be quickly enumerated
from the tag cloud or search field. I should be able to type "restr..." and see
a list of all the restricted processors.

I also think it makes sense to show the shield icon (I was originally going
to suggest a lock icon but I do like the shield) on processors that are already
on the canvas. I think this will also help users who have a processor which we
have now decided is restricted already on their canvas and they suddenly do not
have access to it because of the new policy enforcement.

I am willing to +1 and merge this (I made one small checkstyle correction
in `ITProcessorAccessControl.java` in Javadoc to resolve an issue), but would
like to hear from @mcgilman how much work would be involved in addressing the
minor suggestions I made above. They could be separated into a new enhancement
Jira if necessary, pending release vote timing, as I think the current work
does meet what was discussed in the original Jira.

Thanks Matt.

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

2016-11-19 Thread Andy LoPresto (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-3069:

Priority: Minor  (was: Blocker)

> Error clearing content_repository causes app to fail when configured securely
> -
>
> Key: NIFI-3069
> URL: https://issues.apache.org/jira/browse/NIFI-3069
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>Priority: Minor
>  Labels: content-repository, logging
> Fix For: 1.1.0
>
>
> An issue was introduced into {{master}} (I believe commit 
> [1be08714731f01347ac1f98e18047fe7d9ab8afd|https://github.com/apache/nifi/commit/1be08714731f01347ac1f98e18047fe7d9ab8afd]
>  from [PR 1202|https://github.com/apache/nifi/pull/1202] to resolve 
> [NIFI-2854]) which causes the following line to be repeated in 
> {{logs/nifi-app.log}} infinitely. In a normal instance, the application can 
> still start up successfully, but in a secured instance, this prevents the 
> application from running. 
> {code}
> hw12203:...space/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT
>  (master) alopresto
>  100s @ 17:32:36 $ tail -f logs/nifi-app.log
> 2016-11-18 17:39:03,012 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:04,016 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:05,020 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:06,023 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:07,027 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:08,031 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:09,035 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:10,040 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:11,045 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:12,047 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:13,050 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:14,053 ERROR [Cleanup Archive for default]

[jira] [Commented] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

2016-11-19 Thread Andy LoPresto (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680223#comment-15680223
 ] 

Andy LoPresto commented on NIFI-3069:
-

[~markap14] thanks for that information. I did run the {{ps -aef | grep -i 
nifi}} command yesterday after the {{./bin/nifi.sh status}} said the 
application was not running, and I saw the various java processes that indicate 
the bootstrap was running. I killed those processes at the time, and now if I 
restart the app it seems to work fine. I will lower the severity of this issue, 
as all we need to do is improve the messaging there so people can recover. 

As for the {{illegal key size}} issue you encountered, I do not encounter that, 
as I have the JCE unlimited strength cryptography jurisdiction policy files 
installed. I noticed that it appears you are using a PKCS12 keystore, and there 
is a new known issue [NIFI-3062] I filed where PKCS12 keystores with passwords 
of 8 characters or more will cause that exception on a system with restricted 
cryptographic strength. 

Thanks for your help. 

> Error clearing content_repository causes app to fail when configured securely
> -
>
> Key: NIFI-3069
> URL: https://issues.apache.org/jira/browse/NIFI-3069
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Andy LoPresto
>Priority: Blocker
>  Labels: content-repository, logging
> Fix For: 1.1.0
>
>
> An issue was introduced into {{master}} (I believe commit 
> [1be08714731f01347ac1f98e18047fe7d9ab8afd|https://github.com/apache/nifi/commit/1be08714731f01347ac1f98e18047fe7d9ab8afd]
>  from [PR 1202|https://github.com/apache/nifi/pull/1202] to resolve 
> [NIFI-2854]) which causes the following line to be repeated in 
> {{logs/nifi-app.log}} infinitely. In a normal instance, the application can 
> still start up successfully, but in a secured instance, this prevents the 
> application from running. 
> {code}
> hw12203:...space/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT
>  (master) alopresto
>  100s @ 17:32:36 $ tail -f logs/nifi-app.log
> 2016-11-18 17:39:03,012 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:04,016 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:05,020 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:06,023 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:07,027 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:08,031 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:09,035 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:10,040 ERROR [Cleanup Archive for default] 
> o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for 
> container default due to java.nio.file.NoSuchFileException: 
> /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/nifi-1.1.0-SNAPSHOT/content_repository
> 2016-11-18 17:39:11,045 ERROR [Cleanup Archive for default] 
>

[GitHub] nifi pull request #1250: NIFI-3061 PutAzureEventHub blocks after network int...

2016-11-19 Thread jfrazee

GitHub user jfrazee opened a pull request:

https://github.com/apache/nifi/pull/1250

NIFI-3061 PutAzureEventHub blocks after network interruption

This change maintains status quo and does not enable the existing unit 
tests. The change was tested against 
[NIFI-3061.xml](https://gist.github.com/jfrazee/6820ec854974df88519f99b3388a0874)
 though. As mentioned in 
[NIFI-3070](https://issues.apache.org/jira/browse/NIFI-3070), there isn't a 
good way to provide meaningful unit tests so integration tests should be added 
as a new improvement.

- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?
- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jfrazee/nifi NIFI-3061

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1250.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1250


commit e67548fb804fbfd35dcd77886d47a0c38e0353c0
Author: Joey Frazee 
Date:   2016-11-19T21:25:19Z

Bumped azure-eventhubs version to 0.9.0




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3061) PutAzureEventHub blocks after network interruption

2016-11-19 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679925#comment-15679925
 ] 

ASF GitHub Bot commented on NIFI-3061:
--

GitHub user jfrazee opened a pull request:

https://github.com/apache/nifi/pull/1250

NIFI-3061 PutAzureEventHub blocks after network interruption

This change maintains status quo and does not enable the existing unit 
tests. The change was tested against 
[NIFI-3061.xml](https://gist.github.com/jfrazee/6820ec854974df88519f99b3388a0874)
 though. As mentioned in 
[NIFI-3070](https://issues.apache.org/jira/browse/NIFI-3070), there isn't a 
good way to provide meaningful unit tests so integration tests should be added 
as a new improvement.

- [X] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?
- [X] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
- [X] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jfrazee/nifi NIFI-3061

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1250.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1250


commit e67548fb804fbfd35dcd77886d47a0c38e0353c0
Author: Joey Frazee 
Date:   2016-11-19T21:25:19Z

Bumped azure-eventhubs version to 0.9.0




> PutAzureEventHub blocks after network interruption
> --
>
> Key: NIFI-3061
> URL: https://issues.apache.org/jira/browse/NIFI-3061
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
>
> PutAzureEventHub will block after a network interruption due to a bug in the 
> underlying azure-eventhubs [1] and qpid-proton [2] libraries. This was fixed 
> in proton 0.15.0 and event hubs 0.9.0 so the NiFi dependency needs to be 
> bumped from 0.7 to 0.9.
> The errors are:
> o.a.n.controller.StandardProcessorNode Timed out while waiting for 
> OnScheduled of 'PutAzureEventHub' processor to finish.
> servicebus.trace UnHandled exception while processing events in reactor: 
> java.lang.IllegalArgumentException
> 1. https://github.com/Azure/azure-event-hubs-java/issues/18
> 2. https://issues.apache.org/jira/browse/PROTON-1275



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-3070) Add integration tests for PutAzureEventHub and GetAzureEventHub

2016-11-19 Thread Joey Frazee (JIRA)

Joey Frazee created NIFI-3070:
-

 Summary: Add integration tests for PutAzureEventHub and 
GetAzureEventHub
 Key: NIFI-3070
 URL: https://issues.apache.org/jira/browse/NIFI-3070
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: Joey Frazee
Priority: Minor


There aren't currently any integration tests for PutAzureEventHub or 
GetAzureEventHub and the unit tests are currently inactive (because they're not 
in a src/test/java). Since there isn't a good way to mock Event Hubs, which 
makes the unit tests of limited value, some kind of integration tests should be 
added.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

2016-11-19 Thread Mark Payne (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679727#comment-15679727
 ] 

Mark Payne commented on NIFI-3069:
--

[~alopresto] So I think there are two different things going on here. My guess 
is that you had NiFi already running from that directory when you did a 
complete build (including a {{mvn clean}}). As a result, NiFi continued to run 
but you blew away the bin/, conf/, content_repository/, etc. directories. As a 
result, it is spewing all of the errors because it can't find those files in 
the content_repository/ directory. The reason I think this is what happened is 
that I've done it before (on a few occasions) :) The fact that {{nifi.sh 
status}} indicates that NiFi is not running is a dead giveaway. It says this 
because the run/nifi.status file cannot be found (as a result of mvn clean) and 
as a result, it doesn't know what PID NiFi is using or what Bootstrap port is 
being used. Can you confirm this by running a {{ps -ef | grep nifi}} and see if 
you have a process already running?

Secondly is the issue of NiFi not starting up when running securely. This I am 
able to replicate easily. I see the following in my logs when I try to startup:

{code}
2016-11-19 14:11:59,924 WARN [main] org.eclipse.jetty.webapp.WebAppContext 
Failed startup of context 
o.e.j.w.WebAppContext@339b1c34{/nifi-api,file:///devel/nifi/nifi-assembly/target/nifi-1.1.0-SNAPSHOT-bin/node1/work/jetty/nifi-web-api-1.1.0-SNAPSHOT.war/webapp/,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.1.0-SNAPSHOT.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.1.0-SNAPSHOT.war}
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'requestReplicator': FactoryBean threw exception on object creation; 
nested exception is org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'clusterCoordinator': FactoryBean threw exception 
on object creation; nested exception is 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'clusterCoordinationProtocolSenderListener' defined in class path 
resource [nifi-cluster-protocol-context.xml]: Cannot resolve reference to bean 
'protocolListener' while setting constructor argument; nested exception is 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'protocolListener' defined in class path resource 
[nifi-cluster-protocol-context.xml]: Cannot resolve reference to bean 
'protocolServerSocketConfiguration' while setting constructor argument; nested 
exception is org.springframework.beans.factory.BeanCreationException: Error 
creating bean with name 'protocolServerSocketConfiguration': FactoryBean threw 
exception on object creation; nested exception is java.io.IOException: 
exception decrypting data - java.security.InvalidKeyException: Illegal key size
at 
org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
 ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
 ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585)
 ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
 ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
 ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060)
 ~[spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88)
 ~[na:na]
at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:837)
 ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
 ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:810)
 ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
 ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) 
~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
at

[jira] [Updated] (NIFI-3020) LDAP - Support configurable user identity

2016-11-19 Thread Oleg Zhurakousky (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated NIFI-3020:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> LDAP - Support configurable user identity
> -
>
> Key: NIFI-3020
> URL: https://issues.apache.org/jira/browse/NIFI-3020
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> The current LDAP provider supports a configurable search filter that will 
> allow the user specified login name to be matched against any LDAP entry 
> attribute. We should offer a configuration option that will indicate if we 
> should use the LDAP entry DN or if we should use the login name that was used 
> in the search filter. For instance, this would allow an admin to configure a 
> user to login with their sAMAccountName and subsequently use that name as 
> their user's identity.
> Note: we should default this option to be the user DN in order to ensure 
> backwards compatibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-3020) LDAP - Support configurable user identity

2016-11-19 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679284#comment-15679284
 ] 

ASF subversion and git services commented on NIFI-3020:
---

Commit c8830742eee4470b371246ec9bf14d2fea5fa522 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c883074 ]

NIFI-3020:
- Introducing a strategy for identifying users.
- Fixing issue with the referral strategy error message.
- Adding code to shutdown the application when the authorizer or login identity 
provider are not initialized successfully.

NIFI-3020:
- Updating the admin guide to document the identity strategy.

NIFI-3020:
- Ensuring the request replicator attempts to shutdown regardless of whether 
the flow service properly terminates.

This closes #1236


> LDAP - Support configurable user identity
> -
>
> Key: NIFI-3020
> URL: https://issues.apache.org/jira/browse/NIFI-3020
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> The current LDAP provider supports a configurable search filter that will 
> allow the user specified login name to be matched against any LDAP entry 
> attribute. We should offer a configuration option that will indicate if we 
> should use the LDAP entry DN or if we should use the login name that was used 
> in the search filter. For instance, this would allow an admin to configure a 
> user to login with their sAMAccountName and subsequently use that name as 
> their user's identity.
> Note: we should default this option to be the user DN in order to ensure 
> backwards compatibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1236: LDAP - Configurable strategy to identify users

2016-11-19 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1236


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3020) LDAP - Support configurable user identity

2016-11-19 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679285#comment-15679285
 ] 

ASF subversion and git services commented on NIFI-3020:
---

Commit c8830742eee4470b371246ec9bf14d2fea5fa522 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c883074 ]

NIFI-3020:
- Introducing a strategy for identifying users.
- Fixing issue with the referral strategy error message.
- Adding code to shutdown the application when the authorizer or login identity 
provider are not initialized successfully.

NIFI-3020:
- Updating the admin guide to document the identity strategy.

NIFI-3020:
- Ensuring the request replicator attempts to shutdown regardless of whether 
the flow service properly terminates.

This closes #1236


> LDAP - Support configurable user identity
> -
>
> Key: NIFI-3020
> URL: https://issues.apache.org/jira/browse/NIFI-3020
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> The current LDAP provider supports a configurable search filter that will 
> allow the user specified login name to be matched against any LDAP entry 
> attribute. We should offer a configuration option that will indicate if we 
> should use the LDAP entry DN or if we should use the login name that was used 
> in the search filter. For instance, this would allow an admin to configure a 
> user to login with their sAMAccountName and subsequently use that name as 
> their user's identity.
> Note: we should default this option to be the user DN in order to ensure 
> backwards compatibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1236: LDAP - Configurable strategy to identify users

2016-11-19 Thread olegz

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1236
  
@mcgilman @ijokarumawak squashing and merging


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #1245: Nifi 3066

2016-11-19 Thread gresockj

Github user gresockj commented on the issue:

https://github.com/apache/nifi/pull/1245
  
Hi @markap14 , thanks for submitting this patch.  I tested it out on my 
cluster and it does not appear to resolve the MergeContent problem in my case.  
I'm hoping I deployed the fix correctly: I applied the patch, saw that it only 
affected the nifi-standard-processors, and simply deployed the new version of 
the nifi-standard-nar to my cluster, and restarted the cluster.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

[jira] [Commented] (NIFI-3050) Restrict dangerous processors to special permission

[GitHub] nifi pull request #1247: Introducing restricted components which require add...

[GitHub] nifi issue #1247: Introducing restricted components which require additional...

[jira] [Updated] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

[jira] [Commented] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

[GitHub] nifi pull request #1250: NIFI-3061 PutAzureEventHub blocks after network int...

[jira] [Commented] (NIFI-3061) PutAzureEventHub blocks after network interruption

[jira] [Created] (NIFI-3070) Add integration tests for PutAzureEventHub and GetAzureEventHub

[jira] [Commented] (NIFI-3069) Error clearing content_repository causes app to fail when configured securely

[jira] [Updated] (NIFI-3020) LDAP - Support configurable user identity

[jira] [Commented] (NIFI-3020) LDAP - Support configurable user identity

[GitHub] nifi pull request #1236: LDAP - Configurable strategy to identify users

[jira] [Commented] (NIFI-3020) LDAP - Support configurable user identity

[GitHub] nifi issue #1236: LDAP - Configurable strategy to identify users

[GitHub] nifi issue #1245: Nifi 3066

17 matches

Site Navigation

Mail list logo

Footer information