[
https://issues.apache.org/jira/browse/TEZ-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996994#comment-13996994
]
Rajesh Balamohan edited comment on TEZ-1114 at 5/13/14 10:38 PM:
-----------------------------------------------------------------
For certificates generation: (*Mainly for local VM*; I had all hadoop processes
started as root)
1. export SERVER_KEY_LOCATION=/root/certificates/server/
2. cd $SERVER_KEY_LOCATION ; keytool -genkey -alias tez-vm -keyalg RSA -keysize
1024 -dname "CN=tez-vm,OU=hw,O=hw,L=paloalto,ST=ca,C=us" -keypass password
-keystore server.keystore -storepass password
3. keytool -export -alias tez-vm -keystore server.keystore -rfc -file
server_cert_name -storepass password (replace tez-vm with your hostname)
4. keytool -import -noprompt -alias tez-vm -file server_cert_name -keystore
server.truststore -storepass password
5. keytool -import -noprompt -alias tez-vm -file server_cert_name -keystore
all.jks -storepass password
6. keytool -list -v -keystore all.jks -storepass password
7. chgrp -R root $SERVER_KEY_LOCATION;
8. Restart hadoop services
9. "-Djavax.net.debug=all" in mapred java options will be helpful in debugging
SSL related issues.
In core-site.xml
{code:xml}
<property>
<name>hadoop.ssl.require.client.cert</name>
<value>false</value>
</property>
<property>
<name>hadoop.ssl.hostname.verifier</name>
<value>DEFAULT</value>
</property>
<property>
<name>hadoop.ssl.keystores.factory.class</name>
<value>org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory</value>
</property>
<property>
<name>hadoop.ssl.server.conf</name>
<value>ssl-server.xml</value>
</property>
<property>
<name>hadoop.ssl.client.conf</name>
<value>ssl-client.xml</value>
</property>
{code}
In mapred-site.xml (please change the hardcoded paths according to your
settings)
{code:xml}
<property>
<name>mapreduce.shuffle.ssl.enabled</name>
<value>true</value>
</property>
<property>
<name>mapreduce.shuffle.ssl.file.buffer.size</name>
<value>65536</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx200m -Djavax.net.debug=all</value>
</property>
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx200m -Djavax.net.debug=all</value>
</property>
{code}
ssl-server.xml
{code:xml}
<property>
<name>ssl.server.truststore.location</name>
<value>/root/certificates/server/server.truststore</value>
</property>
<property>
<name>ssl.server.truststore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.server.truststore.type</name>
<value>jks</value>
</property>
<property>
<name>ssl.server.truststore.reload.interval</name>
<value>10000</value>
</property>
<property>
<name>ssl.server.keystore.location</name>
<value>/root/certificates/server/server.keystore</value>
</property>
<property>
<name>ssl.server.keystore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.server.keystore.keypassword</name>
<value>password</value>
</property>
<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
</property>
{code}
ssl-client.xml
{code:xml}
<property>
<name>ssl.client.truststore.location</name>
<value>/root/certificates/server/all.jks</value>
</property>
<property>
<name>ssl.client.truststore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.client.truststore.type</name>
<value>jks</value>
</property>
<property>
<name>ssl.client.truststore.reload.interval</name>
<value>10000</value>
</property>
<property>
<name>ssl.client.keystore.location</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.password</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.keypassword</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.type</name>
<value>jks</value>
</property>
{code}
was (Author: rajesh.balamohan):
In core-site.xml
{code:xml}
<property>
<name>hadoop.ssl.require.client.cert</name>
<value>false</value>
</property>
<property>
<name>hadoop.ssl.hostname.verifier</name>
<value>DEFAULT</value>
</property>
<property>
<name>hadoop.ssl.keystores.factory.class</name>
<value>org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory</value>
</property>
<property>
<name>hadoop.ssl.server.conf</name>
<value>ssl-server.xml</value>
</property>
<property>
<name>hadoop.ssl.client.conf</name>
<value>ssl-client.xml</value>
</property>
{code}
In mapred-site.xml (please change the hardcoded paths according to your
settings)
{code:xml}
<property>
<name>mapreduce.shuffle.ssl.enabled</name>
<value>true</value>
</property>
<property>
<name>mapreduce.shuffle.ssl.file.buffer.size</name>
<value>65536</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx200m -Djavax.net.debug=all</value>
</property>
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx200m -Djavax.net.debug=all</value>
</property>
{code}
ssl-server.xml
{code:xml}
<property>
<name>ssl.server.truststore.location</name>
<value>/root/certificates/server/server.truststore</value>
</property>
<property>
<name>ssl.server.truststore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.server.truststore.type</name>
<value>jks</value>
</property>
<property>
<name>ssl.server.truststore.reload.interval</name>
<value>10000</value>
</property>
<property>
<name>ssl.server.keystore.location</name>
<value>/root/certificates/server/server.keystore</value>
</property>
<property>
<name>ssl.server.keystore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.server.keystore.keypassword</name>
<value>password</value>
</property>
<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
</property>
{code}
ssl-client.xml
{code:xml}
<property>
<name>ssl.client.truststore.location</name>
<value>/root/certificates/server/all.jks</value>
</property>
<property>
<name>ssl.client.truststore.password</name>
<value>password</value>
</property>
<property>
<name>ssl.client.truststore.type</name>
<value>jks</value>
</property>
<property>
<name>ssl.client.truststore.reload.interval</name>
<value>10000</value>
</property>
<property>
<name>ssl.client.keystore.location</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.password</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.keypassword</name>
<value></value>
</property>
<property>
<name>ssl.client.keystore.type</name>
<value>jks</value>
</property>
{code}
For certificates generation: (Mainly for local VM; I had all hadoop processes
started as root.)
1. export SERVER_KEY_LOCATION=/root/certificates/server/
2. cd $SERVER_KEY_LOCATION ; keytool -genkey -alias tez-vm -keyalg RSA -keysize
1024 -dname "CN=tez-vm,OU=hw,O=hw,L=paloalto,ST=ca,C=us" -keypass password
-keystore server.keystore -storepass password
3. keytool -export -alias tez-vm -keystore server.keystore -rfc -file
server_cert_name -storepass password (replace tez-vm with your hostname)
4. keytool -import -noprompt -alias tez-vm -file server_cert_name -keystore
server.truststore -storepass password
5. keytool -import -noprompt -alias tez-vm -file server_cert_name -keystore
all.jks -storepass password
6. keytool -list -v -keystore all.jks -storepass password
7. chgrp -R root $SERVER_KEY_LOCATION;
8. Restart hadoop services
9. "-Djavax.net.debug=all" in mapred java options will be helpful in debugging
SSL related issues.
> DAGs fail with ClassCast exception when encrypted shuffle is enabled.
> ----------------------------------------------------------------------
>
> Key: TEZ-1114
> URL: https://issues.apache.org/jira/browse/TEZ-1114
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Hitesh Shah
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1114-v1.patch
>
>
> The failure should be detected early by the fetcher and a better error
> message generated denoting that ssl is not supported for now.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
