Just wanted to add, from what I have tested so far I see this working fine
with Vanilla Spark reading Iceberg data.
On Wed, May 15, 2019 at 2:59 PM Gautam <gautamkows...@gmail.com> wrote:
> Hello There,
> I am currently doing some testing with Vanilla Spark
> Readers' ability to read Iceberg generated data. This is both from an
> Iceberg/Parquet Reader interoperability and Spark version backward
> compatibility standpoint (e.g. Spark distributions running v2.3.x which
> doesn't support Iceberg DataSource vs. those running 2.4.x) .
>
> To be clear I am talking about doing the following on data written by
> Iceberg :
>
> spark.read.format("parquet").load($icebergBasePath + "/data")
>
> Can I safely assume this will continue to work? If not then what could be
> the reasons and associated risks?
>
> This would be good to know coz these things often come up in migration
> path discussions and evaluating costs associated with generating and
> keeping two copies of the same data.
>
> thanks,
> - Gautam.
>
