date:20150403

[jira] [Created] (HADOOP-11806) Test issue for JIRA automation scripts

2015-04-03 Thread Raymie Stata (JIRA)

Raymie Stata created HADOOP-11806:
-

 Summary: Test issue for JIRA automation scripts
 Key: HADOOP-11806
 URL: https://issues.apache.org/jira/browse/HADOOP-11806
 Project: Hadoop Common
  Issue Type: Test
Reporter: Raymie Stata
Assignee: Raymie Stata
Priority: Trivial


I'm writing some scripts to automate some JIRA clean-up activities.  I've 
created this issue for testing these scripts.  Please ignore...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HADOOP-11740) Combine erasure encoder and decoder interfaces

2015-04-03 Thread Zhe Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhe Zhang resolved HADOOP-11740.

   Resolution: Fixed
Fix Version/s: HDFS-7285

> Combine erasure encoder and decoder interfaces
> --
>
> Key: HADOOP-11740
> URL: https://issues.apache.org/jira/browse/HADOOP-11740
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: io
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
> Fix For: HDFS-7285
>
> Attachments: HADOOP-11740-000.patch, HADOOP-11740-001.patch, 
> HADOOP-11740-002.patch
>
>
> Rationale [discussed | 
> https://issues.apache.org/jira/browse/HDFS-7337?focusedCommentId=14376540&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14376540]
>  under HDFS-7337.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11805) Better to rename some raw erasure coders

2015-04-03 Thread Kai Zheng (JIRA)

Kai Zheng created HADOOP-11805:
--

 Summary: Better to rename some raw erasure coders
 Key: HADOOP-11805
 URL: https://issues.apache.org/jira/browse/HADOOP-11805
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Kai Zheng
Assignee: Kai Zheng


When work on more coders, it was found better to rename some existing raw 
coders for consistency, and more meaningful. As a result, we may have:
XORRawErasureCoder, in Java
NativeXORRawErasureCoder, in native
RSRawErasureCoder, in Java
NativeRSRawErasureCoder, in native and using ISA-L



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11804) POC Hadoop Client w/o transitive dependencies

2015-04-03 Thread Sean Busbey (JIRA)

Sean Busbey created HADOOP-11804:


 Summary: POC Hadoop Client w/o transitive dependencies
 Key: HADOOP-11804
 URL: https://issues.apache.org/jira/browse/HADOOP-11804
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: build
Reporter: Sean Busbey
Assignee: Sean Busbey


make a hadoop-client-api and hadoop-client-runtime that i.e. HBase can use to 
talk with a Hadoop cluster without seeing any of the implementation 
dependencies.

see proposal on parent for details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11803) Extend ECBlock allowing to customize how to read chunks from block

2015-04-03 Thread Kai Zheng (JIRA)

Kai Zheng created HADOOP-11803:
--

 Summary: Extend ECBlock allowing to customize how to read chunks 
from block
 Key: HADOOP-11803
 URL: https://issues.apache.org/jira/browse/HADOOP-11803
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Kai Zheng
Assignee: Kai Zheng


As discussed in HDFS-7715 with [~jack_liuquan] and [~rashmikv], we may need to 
extend {{ECBlock}} in erasure coder layer, allowing erasure codes like 
Hitchhiker to be able to customize the behavior about how to read chunk(s) from 
a block for a coding process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11802) DomainSocketWatcher#watcherThread encounters IllegalStateException in finally block when calling sendCallback

2015-04-03 Thread Eric Payne (JIRA)

Eric Payne created HADOOP-11802:
---

 Summary: DomainSocketWatcher#watcherThread encounters 
IllegalStateException in finally block when calling sendCallback
 Key: HADOOP-11802
 URL: https://issues.apache.org/jira/browse/HADOOP-11802
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Eric Payne


In the main finally block of the {{DomainSocketWatcher#watcherThread}}, the 
call to {{sendCallback}} can encountering an {{IllegalStateException}}, and 
leave some cleanup tasks undone.

{code}
  } finally {
lock.lock();
try {
  kick(); // allow the handler for notificationSockets[0] to read a byte
  for (Entry entry : entries.values()) {
// We do not remove from entries as we iterate, because that can
// cause a ConcurrentModificationException.
sendCallback("close", entries, fdSet, entry.getDomainSocket().fd);
  }
  entries.clear();
  fdSet.close();
} finally {
  lock.unlock();
}
  }
{code}

The exception causes {{watcherThread}} to skip the calls to {{entries.clear()}} 
and {{fdSet.close()}}.

{code}
2015-04-02 11:48:09,941 [DataXceiver for client 
unix:/home/gs/var/run/hdfs/dn_socket [Waiting for operation #1]] INFO 
DataNode.clienttrace: cliID: DFSClient_NONMAPREDUCE_-807148576_1, src: 
127.0.0.1, dest: 127.0.0.1, op: REQUEST_SHORT_CIRCUIT_SHM, shmId: n/a, srvID: 
e6b6cdd7-1bf8-415f-a412-32d8493554df, success: false
2015-04-02 11:48:09,941 [Thread-14] ERROR unix.DomainSocketWatcher: 
Thread[Thread-14,5,main] terminating on unexpected exception
java.lang.IllegalStateException: failed to remove 
b845649551b6b1eab5c17f630e42489d
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at 
org.apache.hadoop.hdfs.server.datanode.ShortCircuitRegistry.removeShm(ShortCircuitRegistry.java:119)
at 
org.apache.hadoop.hdfs.server.datanode.ShortCircuitRegistry$RegisteredShm.handle(ShortCircuitRegistry.java:102)
at 
org.apache.hadoop.net.unix.DomainSocketWatcher.sendCallback(DomainSocketWatcher.java:402)
at 
org.apache.hadoop.net.unix.DomainSocketWatcher.access$1100(DomainSocketWatcher.java:52)
at 
org.apache.hadoop.net.unix.DomainSocketWatcher$2.run(DomainSocketWatcher.java:522)
at java.lang.Thread.run(Thread.java:722)
{code}

Please note that this is not a duplicate of HADOOP-11333, HADOOP-11604, or 
HADOOP-10404. The cluster installation is running code with all of these fixes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11801) Update BUILDING.txt

2015-04-03 Thread Gabor Liptak (JIRA)

Gabor Liptak created HADOOP-11801:
-

 Summary: Update BUILDING.txt
 Key: HADOOP-11801
 URL: https://issues.apache.org/jira/browse/HADOOP-11801
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
Reporter: Gabor Liptak
Priority: Minor


ProtocolBuffer is packaged in Ubuntu



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11800) Clean up some test methods in TestCodec.java

2015-04-03 Thread Akira AJISAKA (JIRA)

Akira AJISAKA created HADOOP-11800:
--

 Summary: Clean up some test methods in TestCodec.java
 Key: HADOOP-11800
 URL: https://issues.apache.org/jira/browse/HADOOP-11800
 Project: Hadoop Common
  Issue Type: Bug
  Components: test
Reporter: Akira AJISAKA


Found two issues when reviewing the patches in HADOOP-11627.
1. There is no {{@Test}} annotation, so the test is not executed.
{code}
  public void testCodecPoolAndGzipDecompressor() {
{code}
2. The method should be private because it is called from other tests.
{code}
  public void testGzipCodecWrite(boolean useNative) throws IOException {
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11799) In Cluster setup documentation ,many of the configurations do not have the default values

2015-04-03 Thread Jagadesh Kiran N (JIRA)

Jagadesh Kiran N created HADOOP-11799:
-

 Summary: In Cluster setup documentation ,many of the 
configurations do not have the default values 
 Key: HADOOP-11799
 URL: https://issues.apache.org/jira/browse/HADOOP-11799
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Jagadesh Kiran N
Priority: Trivial


It would be helpful to have default values for the configuration in Cluster 
Setup documentation page ,so that user can directly use the same for 
installation rather than searching the default values in *defualt.xml. And he 
can modify only required configurations.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HADOOP-11798) Native raw erasure coder in XOR codes

2015-04-03 Thread Kai Zheng (JIRA)

Kai Zheng created HADOOP-11798:
--

 Summary: Native raw erasure coder in XOR codes
 Key: HADOOP-11798
 URL: https://issues.apache.org/jira/browse/HADOOP-11798
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Kai Zheng
Assignee: Kai Zheng


Raw XOR coder is utilized in Reed-Solomon erasure coder in an optimization to 
recover only one erased block which is in most often case. It can also be used 
in HitchHiker coder. Therefore a native implementation of it would be deserved 
for performance gain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)