[
https://issues.apache.org/jira/browse/IGNITE-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ray updated IGNITE-10314:
-------------------------
Affects Version/s: (was: 2.2)
(was: 2.1)
(was: 2.0)
Description:
When user performs add/remove column in DDL, Spark will get the old/wrong
schema.
Analyse
Currently Spark data frame API relies on QueryEntity to construct schema, but
QueryEntity in QuerySchema is a local copy of the original QueryEntity, so the
original QueryEntity is not updated when modification happens.
Solution
Get the schema using sql, get rid of QueryEntity.
was:
When user performs column and index modification operation in SQL(ex create
index, drop index, add column, drop column), QueryEntity in CacheConfiguration
for the modified cache is not updated.
Analyse
QueryEntity in QuerySchema is a local copy of the original QueryEntity, so the
original QueryEntity is not updated when modification happens.
Component/s: spark
Summary: Spark dataframe will get wrong schema if user executes
add/drop column DDL (was: QueryEntity is not updated when column and index
added or dropped)
> Spark dataframe will get wrong schema if user executes add/drop column DDL
> --------------------------------------------------------------------------
>
> Key: IGNITE-10314
> URL: https://issues.apache.org/jira/browse/IGNITE-10314
> Project: Ignite
> Issue Type: Bug
> Components: spark
> Affects Versions: 2.3, 2.4, 2.5, 2.6, 2.7
> Reporter: Ray
> Assignee: Ray
> Priority: Critical
> Fix For: 2.8
>
>
> When user performs add/remove column in DDL, Spark will get the old/wrong
> schema.
>
> Analyse
> Currently Spark data frame API relies on QueryEntity to construct schema, but
> QueryEntity in QuerySchema is a local copy of the original QueryEntity, so
> the original QueryEntity is not updated when modification happens.
>
> Solution
> Get the schema using sql, get rid of QueryEntity.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
