Vladimir Ozerov created IGNITE-3080:
---------------------------------------
Summary: Improve Spark documentation for CDH deployment
Key: IGNITE-3080
URL: https://issues.apache.org/jira/browse/IGNITE-3080
Project: Ignite
Issue Type: Improvement
Components: Ignite RDD
Affects Versions: 1.5.0.final
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 1.6
Users might experience classpath issues when running Spark on CDH with YARN.
The solution is to add Ignite JARs to YARN classpath through CDH console.
Originally explained by Michael Dolgonos here:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Installation-with-Spark-under-CDH-tp4457p4624.html
{quote}
Cloudera Manager -> YARN (MR2 Included) -> Configuration -> Service Wide ->
Advanced -> Spark Client Advanced Configuration Snippet (Safety Valve) for
spark-conf/spark-defaults.conf
Added etc/ignite-fabric-1.5.0/libs/* to already present
spark.executor.extraClassPath=/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/htrace-core-3.2.0-incubating.jar.
The combined line looks like this:
spark.executor.extraClassPath=/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/htrace-core-3.2.0-incubating.jar:/etc/ignite-fabric-1.5.0/libs/*{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
