When I tried to compile the Spark 1.5.2 with -Phive-0.12.0, maven gave me back
an error that profile doesn't exist any more.
But when I read the Spark SQL programming guide here:
http://spark.apache.org/docs/1.5.2/sql-programming-guide.htmlIt keeps
mentioning Spark 1.5.2 still can work with Hive 0.12 meta-store, for example:
Compatibility with Apache HiveSpark SQL is designed to be compatible with the
Hive Metastore, SerDes and UDFs. Currently Hive SerDes and UDFs are based on
Hive 1.2.1, and Spark SQL can be connected to different versions of Hive
Metastore (from 0.12.0 to 1.2.1. Also see
http://spark.apache.org/docs/latest/sql-programming-guide.html#interacting-with-different-versions-of-hive-metastore).
spark.sql.hive.metastore.version1.2.1Version of the Hive metastore. Available
options are 0.12.0 through 1.2.1.
So I am very confusing about this. In our environment, the version of Hadoop
from our vender still comes with Hive 0.12.0. But I plan to upgrade the Spark
(which we deploy by ourselves) from 1.3.1 to 1.5.2. If possible, I wish Spark
1.5.2 still can use the Hive 0.12.0 metadata in Hadoop, but I don't think I
have that option after I tried to compile it.
Is this due to a bug in the document, or am I missing something?
Thanks
Yong
