Thanks R. I had already done that as listed below in the latter part of my question. Wanted to know why the SQL part is giving error.
-Sandeep From: RD <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Saturday, March 30, 2019 at 8:38 AM To: "[email protected]" <[email protected]> Subject: Re: Spark SQL error - Unsupported data source type for direct query on files: iceberg Hi Sandeep, One way would be to register your dataframe, you loaded, as a temporary view. See https://spark.apache.org/docs/latest/sql-getting-started.html#global-temporary-view -R On Fri, Mar 29, 2019 at 5:12 PM Sandeep Sagar <[email protected]<mailto:[email protected]>> wrote: Hi, Exploring Iceberg here. Trying a hello-world in java. Am able to read and write to a table(“company”) on disk with Spark. However, when I tried – Dataset<Row> updated= spark.sql("SELECT * FROM iceberg.company where date_updated != 0"); Got the following exception - Caused by: org.apache.spark.sql.AnalysisException: Unsupported data source type for direct query on files: iceberg; at org.apache.spark.sql.execution.datasources.ResolveSQLOnFile$$anonfun$apply$4.applyOrElse(rules.scala:56) ... 51 more Is it not supported yet or I am doing it incorrectly? I have been able to read the dataset and then filtering on it by using the following (where Company is my POJO) spark.read.format(“iceberg”).load(company.location()).as(Company.getEncoder()); Appreciate the help. -Sandeep The information contained in this email may be confidential. It has been sent for the sole use of the intended recipient(s). If the reader of this email is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this email in error, please notify the sender immediately and destroy all copies of the message. The information contained in this email may be confidential. It has been sent for the sole use of the intended recipient(s). If the reader of this email is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this email in error, please notify the sender immediately and destroy all copies of the message.
