[jira] [Commented] (HADOOP-8468) Umbrella of enhancements to support different failure and locality topologies

[Sheng Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HADOOP-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672764#comment-13672764
 ] 

Sheng Liu commented on HADOOP-8468:
---

Thank you for your work, Junping

 Umbrella of enhancements to support different failure and locality topologies
 -

 Key: HADOOP-8468
 URL: https://issues.apache.org/jira/browse/HADOOP-8468
 Project: Hadoop Common
  Issue Type: Improvement
  Components: ha, io
Affects Versions: 1.0.0, 2.0.0-alpha
Reporter: Junping Du
Assignee: Junping Du
 Attachments: HADOOP-8468-total.patch, HADOOP-8468-total-v3.patch, 
 HVE_Hadoop World Meetup 2012.pptx, HVE User Guide on branch-1(draft ).pdf, 
 Proposal for enchanced failure and locality topologies.pdf, Proposal for 
 enchanced failure and locality topologies (revised-1.0).pdf


 The current hadoop network topology (described in some previous issues like: 
 Hadoop-692) works well in classic three-tiers network when it comes out. 
 However, it does not take into account other failure models or changes in the 
 infrastructure that can affect network bandwidth efficiency like: 
 virtualization. 
 Virtualized platform has following genes that shouldn't been ignored by 
 hadoop topology in scheduling tasks, placing replica, do balancing or 
 fetching block for reading: 
 1. VMs on the same physical host are affected by the same hardware failure. 
 In order to match the reliability of a physical deployment, replication of 
 data across two virtual machines on the same host should be avoided.
 2. The network between VMs on the same physical host has higher throughput 
 and lower latency and does not consume any physical switch bandwidth.
 Thus, we propose to make hadoop network topology extend-able and introduce a 
 new level in the hierarchical topology, a node group level, which maps well 
 onto an infrastructure that is based on a virtualized environment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9435) Native build hadoop-common-project fails on $JAVA_HOME/include/jni_md.h using ibm java

[Sheng Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HADOOP-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628743#comment-13628743
 ] 

Sheng Liu commented on HADOOP-9435:
---

+1 It is good to me. This patch is on a CMakefile, it seems hard to include a 
test. This patch solves the jni_md.h path issue using ibm jdk, and keeps it 
good in oracle jdk. 

 Native build hadoop-common-project fails on $JAVA_HOME/include/jni_md.h using 
 ibm java
 --

 Key: HADOOP-9435
 URL: https://issues.apache.org/jira/browse/HADOOP-9435
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.3-alpha
Reporter: Tian Hong Wang
  Labels: patch
 Fix For: 2.0.3-alpha

 Attachments: HADOOP-9435.patch


 When native build hadoop-common-project with IBM java using command like: 
 mvn package -Pnative
 it will exist the following errors.
  [exec] CMake Error at JNIFlags.cmake:113 (MESSAGE):
  [exec]   Failed to find a viable JVM installation under JAVA_HOME.
  [exec] Call Stack (most recent call first):
  [exec]   CMakeLists.txt:24 (include)
  [exec] 
  [exec] 
  [exec] -- Configuring incomplete, errors occurred!
 The reason is that IBM java uses $JAVA_HOME/include/jniport.h instead of 
 $JAVA_HOME/include/jni_md.h in Oracle java.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira