Hi,
Exploring Iceberg here. Trying a hello-world in java.
Am able to read and write to a table(“company”) on disk with Spark. However,
when I tried –
Dataset<Row> updated= spark.sql("SELECT * FROM iceberg.company where
date_updated != 0");
Got the following exception -
Caused by: org.apache.spark.sql.AnalysisException: Unsupported data source type
for direct query on files: iceberg;
at
org.apache.spark.sql.execution.datasources.ResolveSQLOnFile$$anonfun$apply$4.applyOrElse(rules.scala:56)
... 51 more
Is it not supported yet or I am doing it incorrectly?
I have been able to read the dataset and then filtering on it by using the
following (where Company is my POJO)
spark.read.format(“iceberg”).load(company.location()).as(Company.getEncoder());
Appreciate the help.
-Sandeep
The information contained in this email may be confidential. It has been sent
for the sole use of the intended recipient(s). If the reader of this email is
not an intended recipient, you are hereby notified that any unauthorized
review, use, disclosure, dissemination, distribution, or copying of this
message is strictly prohibited. If you have received this email in error,
please notify the sender immediately and destroy all copies of the message.
