[
https://issues.apache.org/jira/browse/HBASE-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-2080:
----------------------------------
Attachment: HBASE-2080.patch
This is the relevant hunk:
{noformat}
+# Probe for instance volumes
+
+# /dev/sdb as /mnt is always set up by base image
+DFS_NAME_DIR="/mnt/hadoop/dfs/name"
+DFS_DATA_DIR="/mnt/hadoop/dfs/data"
+i=2
+for d in c d e f g h i j k l m n o p q r s t u v w x y z; do
+ m="/mnt${i}"
+ mkdir -p $m
+ mount /dev/sd${d} $m > /dev/null 2>&1
+ if [ $? -eq 0 ] ; then
+ if [ $i -lt 3 ] ; then # no more than two namedirs
+ DFS_NAME_DIR="${DFS_NAME_DIR},${m}/hadoop/dfs/name"
+ fi
+ DFS_DATA_DIR="${DFS_DATA_DIR},${m}/hadoop/dfs/data"
+ i=$(( i + 1 ))
+ fi
+done
{noformat}
> [EC2] Support multivolume local instance storage
> ------------------------------------------------
>
> Key: HBASE-2080
> URL: https://issues.apache.org/jira/browse/HBASE-2080
> Project: Hadoop HBase
> Issue Type: Sub-task
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Fix For: 0.20.3, 0.21.0
>
> Attachments: HBASE-2080.patch
>
>
> The (old) Hadoop EC2 scripts we used as a basis for our scripts do not
> properly handle instance types that have their local storage attached as
> multiple volumes. The AMI build script uses Amazon's latest Fedora 8 AMI as a
> base, which does not automatically mount instance storage on multiple volumes
> either.
> We recommend use of two instance types:
> - c1.medium for zookeeper, which has one vdisk of 340 GB as /dev/sdb mounted
> on /mnt by the base image
> - c1.xlarge for master and slaves, which have four vdisks of 420 GB as
> /dev/sd[bcde], only one of which is mounted on /mnt by the base image
> Additionally, the m1.xlarge instance type, which a user might use anyway, has
> two vdisks of 420 GB as /dev/sd[bc], only one of which is mounted on /mnt by
> the base image.
> The hbase-ec2-init-remote.sh script should probe for all available instance
> storage devices, mount them, and update the DataNode configuration
> appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
