[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436132#comment-15436132
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/915
  
@markap14 Thanks for reviewing the code! @YolandaMDavis Thanks for the 
functional test and detailed test result report!


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436040#comment-15436040
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/915


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436038#comment-15436038
 ] 

ASF subversion and git services commented on NIFI-2525:
---

Commit 671301193bf5b05a42d0e19aaa9503424ec04bc3 in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6713011 ]

NIFI-2525: Fix Proxy auth issue with async send.

Without this fix, NiFi fails to send data via HTTP Site-to-Site through
Proxy which requires authentication due to AsynchronousCloseException.
It happens when async client replays producing contents in order to re-send the
request with auth credential for the proxy server, however the
connection is already closed.
This fix makes NiFi to send actual data only at the second round of requests, 
so that flow-file
contents can be sent without reading it twice.

Signed-off-by: Yolanda M. Davis 

This closes #915


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435987#comment-15435987
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user YolandaMDavis commented on the issue:

https://github.com/apache/nifi/pull/915
  
I setup 2 standalone nifi instances on separate hosts and created a 3rd 
host to serve as proxy server using Apache Web.  NiFi server A was set as the 
push server which generated 10MB Flow Files every 10 seconds. Server A referred 
to server B via a RPG. Server B received input flowfiles and saved them to disk 
via PutFile.  The following test configurations were performed:

1) Unsecured NiFi instances with no Proxy  - Data was exchanged and saved 
on Server B as expected
2) Unsecured NiFi instance with Proxy (Basic/Digest Auth) - when attempting 
to post via Proxy on Server B endpoint nifi-api/data-transfer/input-ports, 
proxy logged connection attempt with Status 407 415 (authentication required, 
post method).  Proxy logged subsequent responses of 201 or 202 for these 
attempts.  For deletes, 407/415  were logged and subsequent 200 was received 
(OK). No exception seen in log on either server. Data exchange completed 
successfully
4) Secured NiFi Instances with no Proxy - After required policy updates in 
NiFi data was exchanged and performed as expected
5)  Secured NiFi Instances with Proxy (Basic/Digest Auth) - Results same as 
test #2 however proxy logs did not detail resource endpoints accessed (I 
believe due to SSL enabled)
6) Secured NiFi  instances with Proxy (No User/Pass Provided) - NiFi logged 
error indicating :  Unable to refresh Remote Group's peers due to response code 
407:Proxy Authentication Required with explanation: null.  Error Message was 
expected and resolved once user/pass provided for proxy.

Given the above tests I believe that proxy authentication is working as 
expected with this patch

+1

Will merge into master shortly





> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434968#comment-15434968
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user YolandaMDavis commented on the issue:

https://github.com/apache/nifi/pull/915
  
@markap14 I can test functionally as well


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434953#comment-15434953
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/915
  
I've reviewed the code here, and from the perspective of a code review I am 
a +1. I have not yet had a chance to test this, though. Will try to get to that 
today. If anyone else has a chance to review functionality, that would be 
great, as well.


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433834#comment-15433834
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/915#discussion_r75969046
  
--- Diff: nifi-assembly/NOTICE ---
@@ -909,6 +909,11 @@ The following binary components are provided under the 
Apache Software License v
 Expert Group and released to the public domain, as explained at
 http://creativecommons.org/publicdomain/zero/1.0/
 
+  (ASLv2) LittleProxy
--- End diff --

@markap14 Thanks for pointing that out. Removed it.


> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2525) HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException when sending through proxy that requires authentication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432127#comment-15432127
 ] 

ASF GitHub Bot commented on NIFI-2525:
--

GitHub user ijokarumawak opened a pull request:

https://github.com/apache/nifi/pull/915

NIFI-2525: Fix Proxy auth issue with async send.

Without this fix, NiFi fails to send data via HTTP Site-to-Site through
Proxy which requires authentication due to AsynchronousCloseException.

It happens when async client replays producing contents in order to re-send 
the
request with auth credential for the proxy server, however the
connection is already closed.

This fix makes NiFi to send actual data only at the second round of 
requests, so that flow-file
contents can be sent without reading it twice.

Unit test cases using LittleProxy are also added to confirm HTTP 
Site-to-Site works with Proxy with/without authentication.

I've tested with Apache mod_proxy and Squid proxy servers with basic/digest 
auth.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijokarumawak/nifi nifi-2525

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/915.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #915


commit 12cf7ef186440ef5de07b04ecaa33d93846ece6f
Author: Koji Kawamura 
Date:   2016-08-23T01:57:57Z

NIFI-2525: Fix Proxy auth issue with async send.

Without this fix, NiFi fails to send data via HTTP Site-to-Site through
Proxy which requires authentication due to AsynchronousCloseException.
It happens when async client replays producing contents in order to re-send 
the
request with auth credential for the proxy server, however the
connection is already closed.
This fix makes NiFi to send actual data only at the second round of 
requests, so that flow-file
contents can be sent without reading it twice.




> HTTP Site-to-Site fails with java.nio.channels.AsynchronousCloseException 
> when sending through proxy that requires authentication
> -
>
> Key: NIFI-2525
> URL: https://issues.apache.org/jira/browse/NIFI-2525
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>
> Pulling data using Remote Process Group from output port works.
> However, pushing data using Remote Process Group to input port fails with 
> AsynchronousCloseException.
> A RPG sends data via POST, then a proxy server returns 407: proxy auth 
> required. After this, the RPG should resend the request with credential, but 
> the data channel is already closed.
> Currently, it uses chunked encoding so that it can stream data to send. 
> Sending actual data twice won't be efficient. We need to do the 
> authentication before start reading flow-file stream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)