alex kamil created PHOENIX-1041:
-----------------------------------
Summary: Building phoenix with hadoop/hbase from CDH
Key: PHOENIX-1041
URL: https://issues.apache.org/jira/browse/PHOENIX-1041
Project: Phoenix
Issue Type: Improvement
Reporter: alex kamil
Priority: Minor
Pls add instructions on building phoenix with hadoop/hbase from CDH. CDH
deployments are very common and it's a nightmare to debug.
This example is for CDH4.6.0: hadoop-2.0.0-cdh4.6.0 and hbase-2.0.0-cdh4.6.0:
1. phoenix/pom.xml changes:
add cloudera repository
<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
2. update hadoop.profile version
<!-- Hadoop Versions -->
<hadoop-one.version>1.0.4</hadoop-one.version>
<!--hadoop-two.version>2.0.4-alpha</hadoop-two.version-->
<hadoop-two.version>2.0.0-cdh4.6.0</hadoop-two.version>
3. update dependency versions
<!-- Dependency versions -->
<!--hbase.version>0.94.15</hbase.version-->
<hbase.version>0.94.15-cdh4.6.0</hbase.version>
<!--hadoop.version>2.0.0</hadoop.version-->
<hadoop.version>2.0.0-cdh4.6.0</hadoop.version>
4. build phoenix with hadoop2 profile
mvn clean install -DskipTests -Dhadoop.profile=2
5. stop hadoop and hbase cluster if running
6. copy phoenix/phoenix-core/target/phoenix-core-3.1.0-snapshot.jar to each
hadoop/hbase node in the cluster (e.g. to /opt/myapps/ directory)
7. add phoenix-core jar to hbase classpath
update /etc/hbase/conf/hbase-env.sh:
export HBASE_CLASSPATH_PREFIX=/opt/myapps/phoenix-core-3.1.0-snapshot.jar
(Note: if I'm using HBASE_CLASSPATH getting SLF4J multiple bindings error, this
is avoided by loading phoenix jar before the rest of the hbase dependencies
with HBASE_CLASSPATH_PREFIX)
8. add phoenix-core jar to hadoop classpath
update /etc/hadoop/conf/hadoop-env.sh (create the file in hadoop conf directory
if doesn't exist)
export HADOOP_CLASSPATH=/opt/myapps/phoenix-core-3.1.0-snapshot.jar
NOTE:
9. start hadoop/hbase cluster
start sqlline:
10. enable DEBUG logging in sqlline log4j.properties
(otherwise sqlline is just "hanging"in case of hadoop version mismatch and
hides the related error: DEBUG: NoSuchMethodError:
org.apache.hadoop.net.NetUtils.getInputStream)
replace WARN and ERROR with DEBUG in phoenix/bin/log4j.properties
11. add commons-collections, hadoop-common and hadoop-auth jars to sqlline
classpath
(Otherwise getting java.lang.NoClassDefFoundError:
org/apache/commons/collections/map/UnmodifiableMap exception)
update phoenix/bin/sqlline.py:
extrajars=
"/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.6.0.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-
cdh4.6.0.jar"
java_cmd = 'java -classpath ".' + os.pathsep +extrajars+ os.pathsep+
phoenix_utils.phoenix_client_jar + \
12. test
phoenix/bin/sqlline.py <my_zookeeper_host>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
