Hi Junjie,
If you can’t load the classes, I’m not sure what’s going on. You should be
able to use --jars and --driver-class-path to add the Jar on the command
line. Try that, and also make sure you’re using a recent Jar built from
master because we renamed from com.netflix a little while ago.
Right now, master only supports reading from HDFS tables in Spark. That
looks like this:
val df = spark.read.format("iceberg").load("hdfs://...")
df.createOrReplaceTempView("table_name")
We’re working on getting better support into Spark with the DataSourceV2
effort. CTAS was just committed, and we will have a PR for reading v2
tables from SQL posted to Spark shortly.
I’m also planning on updating the Spark code so that it can use a Hive
metastore for tables now that we have committed the connection pool in
iceberg-hive.
rb
On Tue, May 14, 2019 at 8:58 PM 俊杰陈 <[email protected]> wrote:
> Try to send this again since I can't see this on dev mail list.
>
> On Tue, May 14, 2019 at 1:24 PM 俊杰陈 <[email protected]> wrote:
>
>> Hi
>>
>> Do we have wiki or doc for spark usage? I tried to put jar in spark jars
>> dir, but when importing org.apache.iceberg.xxxx in spark-shell, it shows
>> iceberg is not a member of package org.apache.
>>
>> --
>> Thanks & Best Regards
>>
>
>
> --
> Thanks & Best Regards
>
--
Ryan Blue
Software Engineer
Netflix
