[jira] [Updated] (MAPREDUCE-6173) Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
[
https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated MAPREDUCE-6173:
-
Resolution: Fixed
Fix Version/s: 2.7.0
Status: Resolved (was: Patch Available)
Document the configuration of deploying MR over distributed cache with
enabling wired encryption at the same time
-
Key: MAPREDUCE-6173
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: distributed-cache, documentation
Affects Versions: 2.6.0
Reporter: Junping Du
Assignee: Junping Du
Fix For: 2.7.0
Attachments: MAPREDUCE-6173-v2.patch, MAPREDUCE-6173.patch, Screen
Shot for MAPREDUCE-6173.png
Use the current documented configuration (specified in
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html)
to work with the cluster enabling shuffle encryption
(http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
will cause the job failed with exception below:
{noformat}
2014-10-10 02:17:16,600 WARN [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to
tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at
com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at
com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
at
sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
{noformat}
This is due to ssl-client.xml is not included in MR tar ball when we deploy
it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job
can resolve the problem and we should document it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6173) Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
[
https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6173:
--
Attachment: MAPREDUCE-6173-v2.patch
Thanks [~xgong] for the review and comments! v2 patch should address the
comment above.
Document the configuration of deploying MR over distributed cache with
enabling wired encryption at the same time
-
Key: MAPREDUCE-6173
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: distributed-cache, documentation
Affects Versions: 2.6.0
Reporter: Junping Du
Assignee: Junping Du
Attachments: MAPREDUCE-6173-v2.patch, MAPREDUCE-6173.patch, Screen
Shot for MAPREDUCE-6173.png
Use the current documented configuration (specified in
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html)
to work with the cluster enabling shuffle encryption
(http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
will cause the job failed with exception below:
{noformat}
2014-10-10 02:17:16,600 WARN [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to
tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at
com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at
com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
at
sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
{noformat}
This is due to ssl-client.xml is not included in MR tar ball when we deploy
it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job
can resolve the problem and we should document it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6173) Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
[
https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6173:
--
Attachment: Screen Shot for MAPREDUCE-6173.png
Document the configuration of deploying MR over distributed cache with
enabling wired encryption at the same time
-
Key: MAPREDUCE-6173
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: distributed-cache, documentation
Affects Versions: 2.6.0
Reporter: Junping Du
Assignee: Junping Du
Attachments: Screen Shot for MAPREDUCE-6173.png
Use the current documented configuration (specified in
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html)
to work with the cluster enabling shuffle encryption
(http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
will cause the job failed with exception below:
{noformat}
2014-10-10 02:17:16,600 WARN [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to
tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at
com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at
com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
at
sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
{noformat}
This is due to ssl-client.xml is not included in MR tar ball when we deploy
it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job
can resolve the problem and we should document it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6173) Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
[
https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6173:
--
Attachment: MAPREDUCE-6173.patch
Document the configuration of deploying MR over distributed cache with
enabling wired encryption at the same time
-
Key: MAPREDUCE-6173
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: distributed-cache, documentation
Affects Versions: 2.6.0
Reporter: Junping Du
Assignee: Junping Du
Attachments: MAPREDUCE-6173.patch, Screen Shot for MAPREDUCE-6173.png
Use the current documented configuration (specified in
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html)
to work with the cluster enabling shuffle encryption
(http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
will cause the job failed with exception below:
{noformat}
2014-10-10 02:17:16,600 WARN [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to
tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at
com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at
com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
at
sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
{noformat}
This is due to ssl-client.xml is not included in MR tar ball when we deploy
it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job
can resolve the problem and we should document it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6173) Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
[
https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6173:
--
Status: Patch Available (was: Open)
Document the configuration of deploying MR over distributed cache with
enabling wired encryption at the same time
-
Key: MAPREDUCE-6173
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: distributed-cache, documentation
Affects Versions: 2.6.0
Reporter: Junping Du
Assignee: Junping Du
Attachments: MAPREDUCE-6173.patch, Screen Shot for MAPREDUCE-6173.png
Use the current documented configuration (specified in
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html)
to work with the cluster enabling shuffle encryption
(http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
will cause the job failed with exception below:
{noformat}
2014-10-10 02:17:16,600 WARN [fetcher#1]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to
tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at
com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at
com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at
com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
at
sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
{noformat}
This is due to ssl-client.xml is not included in MR tar ball when we deploy
it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job
can resolve the problem and we should document it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
