I found this on computer where I built Spark: $ jar tvf /homes/hortonzy/.m2/repository//org/spark-project/hive/hive-exec/0.13.1/hive-exec-0.13.1.jar | grep ParquetHiveSerDe 2228 Mon Jun 02 12:50:16 UTC 2014 org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe$1.class 1442 Mon Jun 02 12:50:16 UTC 2014 org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe$LAST_OPERATION.class 12786 Mon Jun 02 12:50:16 UTC 2014 org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.class
Looking at http://mvnrepository.com/artifact/org.spark-project.hive/hive-exec/0.12.0 , I don't see the package you mentioned. Maybe Patrick can give us more clue. On Sat, Oct 11, 2014 at 7:32 AM, Sadhan Sood <[email protected]> wrote: > > ---------- Forwarded message ---------- > From: Sadhan Sood <[email protected]> > Date: Sat, Oct 11, 2014 at 10:26 AM > Subject: Re: how to find the sources for spark-project > To: Stephen Boesch <[email protected]> > > > Thanks, I still didn't find it - is it under some particular branch ? More > specifically, I am looking to modify the file: ParquetHiveSerDe.java which > is under this namespace: > > org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java and is > being linked through this jar: > > <dependency> > > <groupId>org.spark-project.hive</groupId> > > <artifactId>hive-exec</artifactId> > > <version>0.12.0</version> > > </dependency> > > It seems these are modified versions of the same jar in org.apache.hive: > > <dependency> > > <groupId>org.apache.hive</groupId> > > <artifactId>hive-exec</artifactId> > > <version>0.13.1</version> > > </dependency> > > It'd be great if I can find the sources of this jar so I can modify it > locally and rebuild jar. Not sure if I can rebuild spark with hive-exec > artifact in org.apache.hive without breaking it. > > > > On Fri, Oct 10, 2014 at 7:28 PM, Stephen Boesch <[email protected]> wrote: > >> Git clone the spark project https://github.com/apache/spark.git >> the hive related sources are under sql/hive/src/main/scala >> >> 2014-10-10 16:24 GMT-07:00 sadhan <[email protected]>: >> >> We have our own customization on top of parquet serde that we've been >>> using >>> for hive. In order to make it work with spark-sql, we need to be able to >>> re-build spark with this. It'll be much easier to rebuild spark with this >>> patch once I can find the sources for org.spark-project.hive. Not sure >>> where >>> to find it ? This seems like the place where we need to put our patch. >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-find-the-sources-for-spark-project-tp16187.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> > >
