[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-10 Thread Suresh Srinivas (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549707#comment-13549707
 ] 

Suresh Srinivas commented on HDFS-4353:
---

Colin, can you add high level description of what you have changed in this 
patch?

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: _02a.patch, 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-10 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549959#comment-13549959
 ] 

Colin Patrick McCabe commented on HDFS-4353:


This patch refactors some code in the {{DFSClient}} and the DataNode's 
{{DataXceiver}}.  The refactor encapsulates connections to peers into a single 
class named{{Peer}}.

Suresh, please excuse me if I'm covering things you already know, but I want to 
give some context to random people reading this JIRA.  Java has no standard 
 mechanism for setting write timeouts on blocking sockets.  So we usually 
wrap our sockets in {{org.apache.hadoop.net.SocketOutputStream}}.  This class 
sets the  {{Socket}} to nonblocking and simulates blocking I/O with a 
timeout.  (There is also a parallel 
{{org.apache.hadoop.net.SocketInputStream}}.)  However, we can't *   always* do 
this, since some Sockets cannot be used in non-blocking mode-- for example, the 
SOCKS sockets classes don't support this.  The other thing that we do a   lot 
of is wrapping output and input streams in encrypted streams.

The end result of this is that we end up passing around a lot of objects just 
to represent a single connection to a Peer.  {{IOStreamPair}} is a good example 
of this.  We also end up using {{instanceof}} a lot because we're dealing 
with types that don't have a common ancestor.  This refactor encapsulates all 
of thos objects in a single object, the {{Peer}}.  This avoids the need to use 
{{instanceof}} to set socket timeouts and other properties.

The main reason for doing this refactor now is that {{DomainSocket}}, which is 
introduced by HDFS-4354, doesn't inherit from {{Socket}}.  We made the decision 
not   to inherit from {{Socket}} because inheriting would require us to rely on 
non-public JVM classes.  There is more discsussion on HDFS-347 about this 
issue, if you're curious.

Specific changes:

{{PeerServer}}: a class that creates {{Peers}}.  {{TcpPeerServer}} is basically 
a wrapper around {{ServerSocket}}.  The next patch introduces another subclass, 
 {{DomainPeerServer}}.

{{BlockReader#close}}: now returns the Peer to the PeerCache directly.  This 
replaces the multi-step process involving {{hasSentStatusCode}}, 
{{takeSocket}}, and{{getStreams}}. 

{{SocketCache}}: was renamed to {{PeerCache}}.  Now caches based on 
{{DatanodeID}} rather than socket address.  This is needed to prepare the way 
for puttingDomainSockets into the cache.  Aside from that it should be 
very similar.


 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: _02a.patch, 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-09 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548370#comment-13548370
 ] 

Hudson commented on HDFS-4353:
--

Integrated in Hadoop-Yarn-trunk #91 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/91/])
svn merge -c -1430507 . for reverting HDFS-4353. Encapsulate connections to 
peers in Peer and PeerServer classes (Revision 1430662)
HDFS-4353. Encapsulate connections to peers in Peer and PeerServer classes. 
Contributed by Colin Patrick McCabe. (Revision 1430507)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430662
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/SocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientBlockVerification.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDisableConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java

todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430507
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 

[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-09 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548477#comment-13548477
 ] 

Hudson commented on HDFS-4353:
--

Integrated in Hadoop-Hdfs-trunk #1280 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1280/])
svn merge -c -1430507 . for reverting HDFS-4353. Encapsulate connections to 
peers in Peer and PeerServer classes (Revision 1430662)
HDFS-4353. Encapsulate connections to peers in Peer and PeerServer classes. 
Contributed by Colin Patrick McCabe. (Revision 1430507)

 Result = FAILURE
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430662
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/SocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientBlockVerification.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDisableConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java

todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430507
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 

[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-09 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548493#comment-13548493
 ] 

Hudson commented on HDFS-4353:
--

Integrated in Hadoop-Mapreduce-trunk #1308 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1308/])
svn merge -c -1430507 . for reverting HDFS-4353. Encapsulate connections to 
peers in Peer and PeerServer classes (Revision 1430662)
HDFS-4353. Encapsulate connections to peers in Peer and PeerServer classes. 
Contributed by Colin Patrick McCabe. (Revision 1430507)

 Result = FAILURE
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430662
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/SocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientBlockVerification.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDisableConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java

todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430507
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 

[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-09 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549126#comment-13549126
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12564027/_02a.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3811//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3811//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: _02a.patch, 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-08 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547249#comment-13547249
 ] 

Hudson commented on HDFS-4353:
--

Integrated in Hadoop-trunk-Commit #3194 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/3194/])
HDFS-4353. Encapsulate connections to peers in Peer and PeerServer classes. 
Contributed by Colin Patrick McCabe. (Revision 1430507)

 Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430507
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/SocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/BasicInetPeer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/EncryptedPeer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/Peer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/PeerServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net/TcpPeerServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientBlockVerification.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDisableConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java


 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Fix For: 3.0.0

 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 

[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-08 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547563#comment-13547563
 ] 

Hudson commented on HDFS-4353:
--

Integrated in Hadoop-trunk-Commit #3197 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/3197/])
svn merge -c -1430507 . for reverting HDFS-4353. Encapsulate connections to 
peers in Peer and PeerServer classes (Revision 1430662)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1430662
Files : 
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketInputStream.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketOutputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/SocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/net
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientBlockVerification.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDisableConnCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPeerCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSocketCache.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java


 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is 

[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-07 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546217#comment-13546217
 ] 

Todd Lipcon commented on HDFS-4353:
---

Looking almost ready. A few minor nits:

{code}
+ * it has been configured-- like when we're reading from the last replica
+ * of a block.
{code}
This description is a little unclear to me. Better to say such as when the 
caller has explicitly asked for a file to be opened without checksum 
verification.



{code}
   public static BlockReader newBlockReader(Params params) throws IOException {
+assert(params.getPeer() != null);
+assert(params.getBlock() != null);
+assert(params.getDatanodeID() != null);
{code}

I think {{Preconditions.checkArgument}} is more appropriate here - the checks 
are cheap (likely free) null checks, so we may as well always verify them.



{code}
+  // TODO: create a wrapper class that makes channel-less sockets look like
+  // they have a channel, so that we can finally remove the legacy
+  // RemoteBlockReader.
{code}
Can you reference the JIRA for removing RBR here? I know we have one filed 
somewhere.


+1 after these are addressed.


 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-07 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546246#comment-13546246
 ] 

Todd Lipcon commented on HDFS-4353:
---

Latest patch looks good. +1 pending Jenkins.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-07 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546476#comment-13546476
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563623/02f.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3787//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3787//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-07 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546524#comment-13546524
 ] 

Todd Lipcon commented on HDFS-4353:
---

I'll commit this later tonight or tomorrow morning unless there are any other 
comments.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch, 02f.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-06 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13545652#comment-13545652
 ] 

Colin Patrick McCabe commented on HDFS-4353:


I just ran a few manual tests on this: running with the legacy blockreader, 
running with Kerberos configured, and running with 
{{dfs.encrypt.data.transfer}} set to true.  They all worked.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544189#comment-13544189
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563247/02c.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:red}-1 javac{color:red}.  The patch appears to cause the build to 
fail.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3734//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544292#comment-13544292
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563368/02c.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:red}-1 javac{color:red}.  The patch appears to cause the build to 
fail.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3737//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544511#comment-13544511
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563377/02d.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3738//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3738//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-04 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544568#comment-13544568
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563386/02e.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3740//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3740//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02c.patch, 02c.patch, 
 02-cumulative.patch, 02d.patch, 02e.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread gschen (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542872#comment-13542872
 ] 

gschen commented on HDFS-4353:
--

hi,

how could I unsubscribe from this mailing list?


thank you very much.

justin





 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543397#comment-13543397
 ] 

Todd Lipcon commented on HDFS-4353:
---

{code}
-sock.setSoTimeout(dfsClient.getConf().socketTimeout);
{code}

did this timeout setting (just after the newPeer call) get lost in the refactor?


{code}
+private Peer peer;
+private long time;
{code}
Make these final


{code}
+  private Daemon daemon;
{code}

Put with the other variable declarations (below {{class Value}}



- Rename {{scInstance}} to {{pcInstance}} or just {{instance}}
- Javadoc for {{PeerCache.get(DataNode)}} is out of date - still refers to old 
params and types



{code}
+  public synchronized void put(DatanodeID dnId, 
org.apache.hadoop.hdfs.net.Peer peer) {
{code}
Shouldn't need fully qualified class name here



{code}
-this.socketIn = NetUtils.getInputStream(s);
-this.socketOut = NetUtils.getOutputStream(s, dnConf.socketWriteTimeout);
-this.isLocal = s.getInetAddress().equals(s.getLocalAddress());
+this.socketIn = peer.getInputStream();
+this.socketOut = peer.getOutputStream();
{code}

{{peer.getOutputStream}} here calls {{new SocketOutputStream(channel, 0)}} -- 
ie doesn't set a write timeout.  However, the previous code here did set a 
write timeout. Does this turn out to not matter due to another call later?

I have the same question in a number of other places where we used to pass a 
WRITE_TIMEOUT to {{NetUtils.getOutputStream}} but now appear to end up with 
timeout 0 (eg in {{newBlockReader}}). I see a few explicit calls to 
{{setWriteTimeout}} but can you please double check that we didn't miss any 
cases? Our test coverage of timeout behavior is light.



{code}
+   * Free the resources associated with this peer factory.
+   * This normally includes sockets, etc.
+   *
+   * @throws IOException If there is an error closing the PeerFactory
{code}
Refers to factory when it should say server.



The {{peerServer}} JavaDoc should explain a little bit more about what kind of 
'tracking' it does of connections. I find it a little incongruous that 
{{PeerServer}} is responsible for tracking the set of connected peers. Since 
all connected clients have the same {{Peer}} interface, couldn't it stay as 
part of DataXceiverServer (ie just rename {{childSockets}} to {{childPeers}} or 
{{connectedPeers}}? Also seems strange to me that the {{PeerServer}} interface 
would have {{removePeer}} but not {{addPeer}}.

Along the same lines, it seems odd that {{Peer.close}} would be responsible for 
removing itself from the {{peerServer}} - it's kind of inverted responsibility 
here, and odd since Peers are also used on clients with no associated server.



-{{TcpPeerServer}} missing audience annotation


 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543416#comment-13543416
 ] 

Hadoop QA commented on HDFS-4353:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12563135/02-cumulative.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 7 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 1 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.ha.TestZKFailoverController
  org.apache.hadoop.ha.TestActiveStandbyElectorRealZK
  org.apache.hadoop.ha.TestZKFailoverControllerStress
  org.apache.hadoop.hdfs.TestConnCache
  org.apache.hadoop.hdfs.TestDatanodeDeath
  org.apache.hadoop.hdfs.TestFileAppend2
  
org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
  
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3720//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HDFS-Build/3720//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3720//console

This message is automatically generated.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543522#comment-13543522
 ] 

Colin Patrick McCabe commented on HDFS-4353:


bq. did this timeout setting (just after the newPeer call) get lost in the 
refactor?

Originally: read timeout set in {{newSocket}}, write timeout set in 
{{BlockReaderFactory#newBlockReader}}
Now: both timeouts set in {{BlockReaderFactory#newBlockReader}}

bq. peer.getOutputStream here calls new SocketOutputStream(channel, 0) – ie 
doesn't set a write timeout. However, the previous code here did set a write 
timeout. Does this turn out to not matter due to another call later?

{{DataXceiverServer#run}} sets a write timeout on the peer before passing it to 
{{DataXceiver}}.  However, {{DataXceiver}} plays with the write timeout a lot.  
I suppose we could move the write timeout into {{DataXceiver}} too, just to 
keep them in the same place.

[test failures]

* Looks like I have to put the disable peer cache test into a separate junit 
test, due to our use of a singleton here.

Also, {{LinkedListMultimap}} never calls {{equals}} at all, it seems, so I can 
leave off that part.

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543524#comment-13543524
 ] 

Colin Patrick McCabe commented on HDFS-4353:


er, that should read DataXceiver plays with the *read* timeout a lot

 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HDFS-4353) Encapsulate connections to peers in Peer and PeerServer classes

2013-01-03 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543591#comment-13543591
 ] 

Vinod Kumar Vavilapalli commented on HDFS-4353:
---


Send an email to hdfs-dev-unsubscr...@hadoop.apache.org .

Thanks,
+Vinod





 Encapsulate connections to peers in Peer and PeerServer classes
 ---

 Key: HDFS-4353
 URL: https://issues.apache.org/jira/browse/HDFS-4353
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: datanode, hdfs-client
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: 02b-cumulative.patch, 02-cumulative.patch


 Encapsulate connections to peers into the {{Peer}} and {{PeerServer}} 
 classes.  Since many Java classes may be involved with these connections, it 
 makes sense to create a container for them.  For example, a connection to a 
 peer may have an input stream, output stream, readablebytechannel, encrypted 
 output stream, and encrypted input stream associated with it.
 This makes us less dependent on the {{NetUtils}} methods which use 
 {{instanceof}} to manipulate socket and stream states based on the runtime 
 type.  it also paves the way to introduce UNIX domain sockets which don't 
 inherit from {{java.net.Socket}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira