Hi,
Has anyone successfully integrated Phoenix with Spark on Amazon EMR? I'm using EMR release version emr-5.5.0, which is phoenix 4.9.0 and spark 2.1.0. This is causing me issues when I try to run the below in a zeppelin notebook (from here<https://phoenix.apache.org/phoenix_spark.html>): %spark import org.apache.phoenix.spark._ import org.apache.spark._ val dataSet = List((1L, "1", 1), (2L, "2", 2), (3L, "3", 3)) sc .parallelize(dataSet) .saveToPhoenix( "OUTPUT_TEST_TABLE", Seq("ID","COL1","COL2"), zkUrl = Some("phoenix-server:2181") ) For reference, the only extra configuration I have performed is: * Create the above OUTPUT_TEST_TABLE using the phoenix/bin/sqlline.py localhost on the master node, by running: CREATE TABLE OUTPUT_TEST_TABLE (id BIGINT NOT NULL PRIMARY KEY, col1 VARCHAR, col2 INTEGER); * Added the following to spark-defaults.conf: ** spark.jars file:/usr/lib/phoenix/phoenix-4.9.0-HBase-1.2-client.jar,file:/usr/lib/phoenix/phoenix-core-4.9.0-HBase-1.2.jar When I try to run the above code I get an error saying: java.lang.NoClassDefFoundError: org/apache/spark/Logging. So it seems that phoenix 4.9.0 is compiled against an earlier version of spark (less than 1.5.2)? Is there any way to use spark 2.1 with phoenix 4.9.0? Not really sure why AWS hasn't updated to Phoenix 4.10.0 yet? Thanks, Steve Robinson steve.robin...@aquilainsight.com 0131 290 2300 [https://aquilainsight.sharepoint.com/Phoenix/_layouts/15/guestaccess.aspx?guestaccesstoken=coZwd1lxzVikDFySR9HQQUgSdTBDtR24zCE7VI3ucKE%3d&docid=0c609694b07114f03a01631f3c5cc4606&rev=1][https://aquilainsight.sharepoint.com/Phoenix/_layouts/15/guestaccess.aspx?guestaccesstoken=XWaweiSSd7YO1IFgfwqm3AAn7KKCsmBf%2f73IlT3d0zE%3d&docid=0cea80d160d954b9584aef7090a5c4ef5&rev=1][https://aquilainsight.sharepoint.com/Phoenix/_layouts/15/guestaccess.aspx?guestaccesstoken=hTrHq%2fmrgDTxOp4jWXzYVM04wsasy2aNJfmG6EJJ%2f9g%3d&docid=00a6dd20560b1438fadebfb8a1255be41&rev=1] www.aquilainsight.com<http://www.aquilainsight.com> [https://aquilainsight.sharepoint.com/Phoenix/_layouts/15/guestaccess.aspx?guestaccesstoken=N79xtBiBY4r5ry1TCu0P%2bce%2f%2b3HFTwwamnQ47PieOoo%3d&docid=03f7d1040c43f4fa0bcdf7f17fa89dfcc&rev=1]linkedin.com/aquilainsight<https://www.linkedin.com/company/aquila-insight> [https://aquilainsight.sharepoint.com/Phoenix/_layouts/15/guestaccess.aspx?guestaccesstoken=fdX1gHdkBdEZ%2bOap1Nr7kTrjMoFxgTZI4RfHFw0R7mw%3d&docid=0869faaa87f6c402fa845a320c225e213&rev=1]twitter.com/aquilainsight<http://twitter.com/aquilainsight>
